HDECOM 


Next-Generation 

NATO  Reference  Mobility  Model  (NG-NRMM) 
Final  Report  by  NATO  Exploratory  Team  ET-148 

Editors 

Jean  Dasch,  Alien  Science  and  Technology,  USA 
Paramsothy  Jayakumar,  US  Army  TARDEC,  US 

Authors 

Michael  Bradbury,  Defence  Science  &  Technology  Laboratory,  UK 
Jean  Dasch,  Alion  Science  and  Technology,  USA 
Ramon  Gonzalez,  MIT,  USA/Spain 
Henry  Hodges,  Nevada  Automotive  Test  Center,  USA 
Abhinandan  Jain,  Jet  Propulsion  Laboratory,  USA 
Karl  lagnemma,  MIT,  USA 
Michael  Letherwood,  US  Army  TARDEC,  USA 
Michael  McCullough,  BAE,  USA 
Jody  Priddy,  US  Army  ERDC,  USA 

Brian  Wojtysiak,  US  Army  Materiel  Systems  Analysis  Activity  (AMSAA),  USA 
J.Y.  Wong,  Vehicle  Systems  Development  Corp.,  Canada 

ET-148  Leaders 

Paramsothy  Jayakumar,  US  Army  TARDEC,  USA 
Michael  Hoenlinger,  Kraus-Maffei  Wegmann  GmbH&Co,  Germany 

AVT  Panel  Board  Member  Sponsor 

David  Gorsich,  US  Army  TARDEC,  USA 


Disclaimer:  Reference  herein  to  any  specific  commercial  company,  product,  process,  or  service  by  trade  name,  trademark,  manufacturer, 
or  otherwise,  does  not  necessarily  constitute  or  imply  its  endorsement,  recommendation,  or  favoring  by  the  United  States  Government  or 
the  Department  of  the  Army  (DoA).  The  opinions  of  the  authors  expressed  herein  do  not  necessarily  state  or  reflect  those  of  the  United 
States  Government  or  the  DoA.  and  shall  not  be  used  for  advertising  or  product  endorsement  purposes. 


UNCLASSIFIED:  Distribution  Statement  A.  Approved  for  public  release;  distribution  is  unlimited 


Final  Report  of  Exploratory  Team,  ET-148 

Next  Generation  NATO  Reference  Mobility  Model 

(NRMM)  Development 

EXECUTIVE  SUMMARY 


The  NATO  Reference  Mobility  Model  (NRMM)  is  a  simulation  tool  aimed  at  predicting  the  capability  of  a 
vehicle  to  move  over  specified  terrain  conditions.  NRMM  was  developed  and  validated  by  the  U.S.  Army 
Tank  Automotive  Research,  Development,  and  Engineering  Center  (TARDEC)  and  Engineer  Research  and 
Development  Center  (ERDC)  in  the  1960s  and  ‘70s,  and  has  been  revised  and  updated  throughout  the  years, 
resulting  in  the  most  recent  version,  NRMM  II.  NRMM  is  traditionally  used  to  facilitate  comparisons  between 
vehicle  design  candidates  and  to  assess  the  mobility  of  existing  vehicles  under  specific  scenarios. 

Although  NRMM  has  proven  to  be  of  great  practical  utility  to  the  NATO  forces,  when  compared  to  modem 
modeling  tools  it  exhibits  several  inherent  limitations.  It  is  based  on  empirical  observations,  and  therefore 
extrapolation  outside  of  test  conditions  is  difficult  or  impossible.  It  is  heavily  dependent  on  in-situ  soil 
measurements.  Only  two-dimensional  analysis  is  possible;  lateral  vehicle  dynamics  are  not  considered.  It 
does  not  account  for  vehicle  dynamic  effects,  but  instead  only  considers  steady-state  conditions.  It  is  specific 
to  wheeled/tracked  vehicles.  It  is  not  easily  implementable  within  modern  vehicle  dynamics  simulations.  It 
exhibits  poor  (or  poorly  understood)  inter-operability  and  inter-scalability  with  other  terramechanics  and  soil 
mechanics  models. 

Exploratory  Team  148  was  formed  to  explore  the  development  of  a  Next-Generation  NRMM  (NG-NRMM). 
Theme  areas  were  developed  and  teams  worked  on  Requirements,  Methodology,  Tool  Choices,  and 
Input/Output  needs  for  a  NG-NRMM.  Two  new  areas  were  also  explored  that  were  not  part  of  the  original 
NRMM:  stochastics  and  intelligent  vehicles.  Based  on  the  results  of  the  exploration  of  tool  choices,  a 
benchmarking  exercise  was  also  planned  to  understand  the  capabilities  of  the  physics-based  tools  available 
from  software  developers. 

Through  this  effort,  the  goal  is  to  have  a  mobility  model  with  enhanced  capabilities  in  the  following  areas: 

•  Increased  flexibility  to  support  operations  by  assessing  the  operational  mobility  of  different  deployed  platforms 
in  different  areas  of  operation  and  routes 

•  Improved  flexibility  as  a  design  and  procurement  support  tool  through  enhanced  fidelity  and  the  ability  to  model 
current  and  emerging  mobility  technologies 

At  the  conclusion  of  ET-148,  the  committee  consisting  of  38  persons  from  13  nations,  was  confident  that  the  time 
was  right  to  develop  an  improved  vehicle  mobility  model  appropriate  to  the  needs  of  the  NATO  nations.  As  laid 
out  in  this  report,  the  requirements  and  methodology  necessary  for  developing  a  NG-NRMM  have  been  well 
specified.  The  follow-on  activity,  AVT-248,  has  been  approved  and  will  proceed  from  2016  to  2018  to  develop 
such  a  model. 
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Chapter  1  -  INTRODUCTION 


1.1  BACKGROUND 

The  NATO  Reference  Mobility  Model  (NRMM)  is  the  accepted  international  standard  for  modeling  the 
mobility  of  ground  combat  and  tactical  vehicles.  It  is  a  simulation  tool  aimed  at  predicting  the  comparative 
capability  of  a  vehicle  to  move  over  specified  terrain.  NRMM  can  be  used  for  on-road  and  cross-country 
scenarios,  and  it  can  account  for  several  parameters  such  as  terrain  type  moisture  content,  terrain  roughness,  and 
vehicle  geometry. 

The  model  was  originally  developed  and  validated  in  the  USA  in  the  1970s  by  the  U.S.  Army  Tank  Automotive 
Research,  Development,  and  Engineering  Center  (TARDEC)  in  Warren,  MI  and  the  US  Corps  of  Engineers 
Waterways  Experimental  Station  (WES)  in  Vicksburg,  MS.  The  Engineer  Research  and  Development  Center 
(ERDC)  remains  the  code  custodian  and  is  responsible  for  configuration  control. 

NRMM  has  proven  of  great  practical  value  to  the  NATO  nations  since  its  development  in  the  70s.  Although  it 
has  been  revised  over  the  years,  the  basis  of  NRMM  is  40  years  old.  When  compared  to  modem  modeling  tools, 
it  exhibits  inherent  limitations;  primarily: 

•  It  is  heavily  dependent  on  empirical  observations  such  as  in-situ  soil  measurements  so  that  extrapolation 
outside  of  test  conditions  is  difficult. 

•  Only  two-dimensional  analysis  is  possible. 

•  It  does  not  account  for  vehicle  dynamic  effects;  rather  it  only  considers  steady-state  conditions  for  cross¬ 
country  mobility. 

•  It  is  not  easily  implemented  with  modem  vehicle  dynamics  simulations  or  other  terramechanics  models. 

•  It  does  not  address  uncertainty. 

•  It  does  not  account  for  the  different  drivers  and  constraints  associated  with  unmanned  ground  vehicles  or 
alternate  vehicle  control  strategies. 

1.2  PURPOSE 

Due  to  the  recognition  of  the  need  for  an  updated  model,  a  NATO  Exploratory  Team  was  proposed  during  the 
spring  2014  NATO  AVT  meeting  in  Copenhagen,  Denmark  by  Panel  Member  Dr.  David  Gorsich,  Chief 
Scientist  of  TARDEC.  The  scope  was  to  investigate  an  efficient  simulation-based  next-generation  NRMM. 
Specifically  the  objectives  were  as  follows  [TAP,  2014]: 

•  Identify  scale-invariant  terrain  descriptions  for  representing  topographic  map  data  (obtained  at  various 
scales)  within  a  suitable  multi-body  dynamic  simulator.  This  will  enable  automated  analysis  of  regions  of 
interest,  given  heterogeneous  map  data  products  as  inputs. 

•  Develop  efficient,  automated,  parallelizable  experimental  design  methods  (i.e.  sampling  methods)  for 
extracting  metrics  of  interest  from  Monte  Carlo  simulations  of  the  multi-body  dynamic  simulator,  including 
mobility-related  metrics  and  auxiliary  metrics.  This  will  yield  rich  statistical  mobility-related  outputs  in  a 
computationally  efficient  manner,  which  will  allow  use  of  modem  HPC  resources. 

•  Explore  the  use  of  compact  representations  of  vehicle  dynamics  (i.e.  response  surface  methods  or  other 
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approximation  methods)  within  the  multi-hody  dynamic  simulator,  with  a  goal  of  further  reducing 
computational  cost. 

•  Establish  compact,  user-friendly  representations  of  output  metrics  that  capture  important  dependencies.  This 
will  yield  an  update  to  classical  “speed  made  good”  or  “go/no  go”  maps. 

The  Exploratory  Team,  as  described  in  the  Technical  Activity  Proposal  (TAP),  was  approved  by  the  AVT  Panel 
under  the  designation  ET-148,  Next-Generation  NRMM  Development.  The  TAP  for  ET-148  is  included  in 
Appendix  A. 

1.3  ENHANCED  CAPABILITIES 

Through  this  effort,  the  goal  is  to  have  a  mobility  model  with  enhanced  capabilities  as  in  the  examples  below: 

•  Increased  flexibility  to  support  operations  by  assessing  the  operational  mobility  of  different  deployed 
platforms  in  different  areas  of  operation  and  routes 

•  Improved  flexibility  as  a  design  and  procurement  support  tool  through  enhanced  fidelity  and  the  ability 
to  model  current  and  emerging  mobility  technologies 

1.4  REFERENCES 

Technical  Activity  Proposal  2014.  Next-Generation  NATO  Reference  Mobility  Model  (NRMM) 
Development,  Activity  Reference  Number  P-2014-30. 
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Chapter  2  -  ORGANIZATION 


2.1  ET-148  ORGANIZATION 

TARDEC  initiated  the  formation  of  ET-148  at  the  spring  2014  NATO  meeting  in  Copenhagen,  Denmark  with 
Dr.  Paramsothy  Jayakumar  of  TARDEC  as  the  Chairperson  and  the  United  States  as  the  lead  nation.  Dr. 
Michael  Hoenlinger  of  Germany  was  later  named  as  the  Co-Chair. 

Starting  in  June  of  2014,  the  group  held  monthly  teleconferences  through  the  end  of  2015.  At  the  first  June  2014 
teleconference,  the  memhership  had  already  grown  to  26  members  from  1 1  nations  (Canada,  Czech  Republic, 
Estonia,  Germany,  Italy,  Poland,  Romania,  Slovakia,  Turkey,  United  Kingdom,  and  the  United  States).  By  fall 
of  2015,  the  membership  had  grown  further  to  38  members  from  13  nations. 

In  addition  to  the  monthly  teleconferences,  the  group  physically  met  three  times,  in  Brussels,  Belgium  from 
October  13-17,  2014,  in  Rzeszow,  Poland  from  April  20-24,  2015  and  in  Prague,  Czech  Republic  from  October 
12-16,  2015.  The  three  meetings  were  attended  by  21  members  from  9  nations,  21  members  from  10  nations, 
and  22  members  from  10  nations,  respectively. 

The  overall  project  was  divided  into  seven  theme  areas,  each  with  a  theme  lead.  All  of  the  members  of  ET-148 
selected  one  or  more  theme  teams  to  join,  depending  on  their  interest  and  area  of  expertise.  The  seven  theme 
areas  and  their  leads  were: 


Theme  1:  Requirements 

Theme  2:  Methodology 

Theme  3:  Stochastics 

Theme  4:  Intelligent  Vehicle 

Theme  5:  Tool  Choices 

Theme  6:  Input  Data  and  Output  Metrics 

Theme  7:  Verification  and  Validation 


Jody  Priddy/Michael  Bradbury 

Mike  McCullough 

Karl  lagnemma,  Ramon  Gonzalez 

Abhi  Jain 

Henry  Hodges 

Brian  Wojtysiak 

Michael  Letherwood 
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Chapter  3  -  NRMM  HISTORY 

Jean  Dasch 


3.1  HISTORY 

Mobility  modeling  began  in  tbe  US  to  address  vehicle  shortcomings  recognized  during  World  War  II.  Vehicle- 
terrain  testing  labs  were  set  up  with  extensive  test  facilities  at  the  United  States  Army  laboratories,  WES  [Jones, 
201 1]  and  the  TARDEC  Land  Locomotion  Laboratory  [Liston,  1965].  Lollowing  decades  of  research,  the  Army 
Materiel  Command  requested  that  the  two  Army  Labs  (TARDEC,  WES)  work  together  on  a  mobility  model. 
The  two  labs  in  coordination  with  Stevens  Institute  of  Technology  issued  the  AMC-71  Mobility  Model  in  1971 
[AMC  ’71,  1973)].  As  described  in  the  Foreword  to  the  report  on  the  model,  “mathematical  modeling  allows 
for  the  evaluation  of  the  entire  vehicle  system  (engine,  transmission,  suspension,  weight,  geometry,  inertia, 
winching  capacity,  and  so  on)  as  it  interacts  with  soil,  vegetation,  slopes,  ditches,  mounds  and  other  features  in  a 
synergistic  fashion.” 

Three  years  of  verification  followed  using  three  vehicle  types  at  five  test  sites  with  the  result  that  AMC-7 1  was 
considered  to  be  correct  about  70%  of  the  time  [Schreiner  &  Willoughby,  1976].  A  refined  model  was  issued  in 
1974  known  as  AMC-74  with  improved  terrain  quantification  and  vehicle-terrain  interactions.  Meanwhile  in 
1976,  NATO  AC/225  Panel  11,  which  was  part  of  the  NATO  Army  Armament  Group  (NAAG),  recognized  the 
need  for  standardized  techniques  to  compare  vehicle  performance  and  the  US  offered  to  help  initiate  this  effort 
[Haley  et  ah,  1979).  This  was  accepted  by  Panel  II  and  AC  225/Working  Group  I  (WGI)  was  established  with 
membership  from  six  countries  (Canada,  Prance,  Germany,  the  Netherlands,  the  United  Kingdom,  and  United 
States)  and  the  first  meeting  was  held  at  TARDEC  in  1977. 

US  members  from  TARDEC,  Peter  Haley,  and  Stevens,  Peter  Jurkat,  visited  each  of  the  six  nations  to  ensure 
that  they  had  the  model  running  correctly  on  their  computers.  The  NATO  working  group  recommended  to  Panel 
n  that  a  Technical  Management  Committee  (TMC)  be  formed  and  this  was  done  in  1978  with  the  same  six 
member  nations  and  led  by  Mr.  Zoltan  Janosi  of  TARDEC.  They  met  regularly  to  bring  participating  countries 
up  to  speed  on  tbe  model  and  to  continue  to  update  tbe  model  as  needed.  The  model  was  accepted  by  NATO  as 
a  reference  model  in  1978  and  was  called  the  Initial  NATO  Reference  Mobility  Model  (INRMM)  and  later  the 
“Initial”  was  dropped  leaving  NRMM.  It  was  also  added  to  U.S.  military  vehicle  specifications  to  ensure  that 
contractors  used  the  model  to  meet  vehicle  requirements,  guaranteeing  wide  usage  of  the  model  [Petrick  et  al, 
1981]. 

Research  and  development  continued  and  the  second  version  of  the  model,  NRMM  11,  was  issued  in  1992 
incorporating  many  of  the  changes  that  were  made  in  the  interim  [Ahlvin  and  Haley,  1992].  The  new  algorithms 
were  mainly  due  to  the  mobility  tests  conducted  by  WES  since  1979  including  the  wheeled  vs  tracked  test 
program  (Willoughby  et  al,  1991)  and  included  new  equations  in  tbe  area  of  soil  traction,  soil  resistance,  and 
surface  slipperiness.  In  addition,  special  software  was  included  to  encompass  radial  tires  and  central  tire 
inflation  systems  (CTIS). 
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All  changes  to  the  model  had  to  he  approved  through  the  TMC.  The  TMC  was  disbanded  in  1997,  hut  each  of 
the  participating  nations  continued  to  advance  their  mobility  modeling  technology  independently,  leading  to  a 
duplication  of  effort.  There  was  a  need  to  reassemble  the  international  community  to  consolidate  these 
independent  and  often  duplicative  efforts  into  a  collection  of  tools  that  would  be  considered  a  new  version  of 
NRMM  and,  subsequently  to  validate,  standardize  and  maintain  the  resulting  package  as  a  shared  NATO 
resource.  Dr.  Richard  McClelland,  TARDEC  Director,  proposed  the  idea  to  the  NATO  Applied  Vehicle 
Technology  (AVT)  panel  in  the  fall  of  2002  [McClelland,  2002].  The  NATO  AVT-107  -  Mobility  Modelling 
Working  Group  was  set  up  to  coordinate  and  conduct  this  task.  AVT-107  first  met  in  October  2002  and 
concluded  in  2006,  with  eight  meetings  held  in  the  interim.  The  primary  countries  involved  were  Canada, 
France,  Romania,  the  United  Kingdom  and  the  United  States  with  lesser  involvement  by  the  Netherlands  and 
Germany. 

At  the  time  of  AVT-107,  a  Vehicle  Terrain  Interface  (VTI)  code  was  built  in  the  US  as  a  result  of  the  Joint  Army 
High-Resolution  Ground  Vehicle  and  Terrain  Mechanics  Program  (HGTM)  by  ERDC,  TARDEC  and  the  Army 
Research  Laboratory  (ARE)  [Richmond  et  ah,  2004;  Lamb  et  ah,  2003;  Reid  et  ah,  2007].  A  number  of  studies 
followed  to  investigate  and  validate  the  VTI  code  [e.g.,  Romano  and  Schultz,  2004;  Parker  et  ah,  2009]. 
Meanwhile,  the  French  had  developed  their  own  code  for  modeling  vehicle  dynamics  that  was  validated  and 
tested,  known  as  PROSPER,  which  could  do  all  the  calculations  done  by  VEHDYN  II.  [Schafer  and  Andre, 
1997]  Eventually  these  new  methodologies  were  not  incorporated  into  NRMM,  either  due  to  confidentiality  or 
commercial  restrictions  [Shoop,  2016].  The  results  from  AVT-107  were  presented  to  the  AVT  Panel  on  6 
October  2006  [AVT,  2006]  and  the  final  report  was  published  in  2011  [Jones  et  al.  2011].  The  committee’s 
work  and  the  final  report  are  valuable  in  several  respects  in  that  the  following  areas  are  extensively  discussed: 

•  A  history  of  the  development  of  the  NRMM  model  from  the  1960s. 

•  A  detailed  status  of  the  model 

•  Identified  limitations 

•  Communication  of  NRMM  usage  and  upgrades  by  various  nations 

Despite  the  successes  of  AVT-107,  many  of  the  NRMM  tool  limitations  were  eventually  not  addressed.  As  a 
result,  NRMM  is  less  effectively  used  by  the  NATO  nations.  One  significant  concern  is  that  if  the  current  tool  is 
not  enhanced  with  higher  fidelity  and  efficiency,  it  will  leave  the  NATO  nations  with  a  subpar  mobility  tool  that 
is  neither  capable  of  accurately  differentiating  competing  designs  nor  capable  of  accurately  predicting  mobility 
performance  of  a  specific  design  in  various  operational  scenarios. 
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Chapter  4  -  NRMM  OVERVIEW 

Michael  Bradbury 


4.1  NRMM  METHODOLOGY 

NRMM  ...  can  realistically  quantify  ground  vehicle  mobility  based  on  terrain  accessibility  and 
maximum  attainable  speeds  for  comparative  force  projection  assessments  of  military  vehicles  via 
rational  consideration  of  the  vehicle's  mission,  design  characteristics,  and  actual  terrain 
characteristics  around  the  globe.  Jody  Priddy,  ERDC,  2014 

NRMM  is  a  modeling  suite  comprising  obstacle  crossing  and  ride  pre -processors  feeding  into  a  main 
(predictions)  module;  the  pre -processors  are  employed  to  reduce  computational  overhead.  Each  of  these  three 
models  requires  different  parameters  of  terrain,  vehicle  and  scenario  (or  control)  data. 
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Figure  4-1.  NRMM  Methodology 


The  submodules  in  turn  contain  sub-models  that  each  considers  specific  aspects  of  mobility  performance.  These 
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include:  obstacle  override  and  avoidance,  vegetation  override  and  performance,  powertrain  performance, 
vehicle/surface  interface  (soils  and  hard  surfaces),  slope  effects  (grades  and  side  slopes),  ride  dynamics, 
visibility,  tire  constraints,  road  curvature  and  braking.  Note  that  in  newer  versions,  Vehdyn  II  and  OBSDP  are 
combined  into  VEHDYN  4.0  along  with  many  other  enhancements. 

NRMM  considers  the  entire  vehicle  underbody  profile  to  check  for  obstacle  interference,  but  only  half  the 
vehicle  for  speed  predictions  (bicycle  model).  In  addition,  only  vertical  acceleration  is  considered  as  a  criteria 
for  ride  dynamics;  the  model  only  considers  steady  state  speed  and  not  acceleration  or  deceleration  within  the 
terrain  unit.  Also,  the  model  cannot  consider  soil  discontinuities  such  as  rocks  or  the  complete  impact  of 
vegetation. 


4.2  PREPROCESSORS  (OBSDP  AND  VEHDYN) 

OBS78b  is  the  obstacle  crossing  pre -processor  for  NRMM.  It  places  a  vehicle  statically  and  sequentially  along  a 
terrain  profile,  and  at  each  point  it  records  the  minimum  clearance  and  the  tractive  effort  required  to  hold  the 
vehicle  in  place.  The  output  of  the  model  is  a  lookup  table,  usually  based  on  72  standard  obstacles,  providing 
minimum  clearance,  maximum  and  average  tractive  effort.  This  lookup  table  forms  part  of  the  vehicle  input 
data  set  for  the  main  module  and  is  used  to  interpolate  results  for  the  unique  obstacles  within  the  main  module’s 
terrain  data. 

It  is  a  two-Dimensional  model  (viewed  from  the  side)  representing  any  given  vehicle  as  front  and  rear 
assemblies  (single  or  paired  axles).  Wheeled  vehicles  can  also  include  a  single  assembly  trailer;  tracked  vehicles 
include  sprocket  and  idler. 

However,  OBSDP  assumes  that  the  tire  is  rigid  and  that  the  ground  clearance  for  the  under  vehicle  profile  is 
fixed  whereas  actual  vehicle  suspensions  allow  for  suspension  droop  and  jounce  and  cause  the  under  vehicle 
profile  to  change  dynamically. 
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Figure  4-2.  Vehicle  configurations 


Figure  4-3.  Terrain  representation 

The  VEHDYN  model  was  originally  developed  in  1974  to  provide  ride  and  shock  simulation  capability  for 
general  use  in  support  of  what  was  then  the  Army  Mobility  Model  now  known  as  NRMM.  Since  then  it  has 
been  revised  over  the  years  and  is  now  known  as  VEHDYN4.0.  VEHDYN4.0  is  a  2-dimensional  model  of  a 
vehicle  that  includes  improved  track  tension,  direct  user-input  setting  configuration,  full  hysteretic  rotational 
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springs  in  both  the  bogie  and  walking  beam  models  and  enhanced  outputs. 

VEHDYN  is  used  to  assess  both  obstacle  impact  (usually  2.5g  vertical  acceleration)  and  ride  (usually  6  Watts 
absorbed  power)  driven  speed  limitations.  These  are  used  to  temper  platform  performance  by  crew  tolerance. 


Figure  4-4.  Generic  VEHDYN  constraint  curve 


4.3  INPUT  REQUIREMENTS 

NRMM  requires  a  broad  and  detailed  set  of  data.  The  data  falls  into  four  types:  scenario,  terrain,  vehicle  and 
operator.  Some  terrain  information  can  be  input  in  either  the  scenario  file  or  the  terrain  file.  A  partial  list  of 
variables  in  the  three  main  categories  is  given  below.  A  fuller  description  is  given  in  Chapter  11. 
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Scenario  data 

Terrain  data 

Vehicle  input 

Snow  depth  and  density 

Surface  condition,  e.g.  normal. 

General  dimensions 

Freeze  and/or  thaw  depth 

slippery 

Axles,  bogies  or  track  assemblies 

Driver:  maximum  braking 

uses  soil  type  classification 

Number  of  powered  or  braked 

acceleration,  braking  reaction 

Land  use 

assemblies 

time,  safety  factor,  recognition 
distance 

Wetness  index 

Pushbar  height  and  force 

Plowing  depth 

Soil  strength:  0-6”,  6-12”,  data  for 

Driver’s  position,  eyes  and  seat 

Seasonal  visibility 

four  ‘seasons’ 

Center  of  gravity 

Obstacles:  height,  width,  length. 

Depth  to  bedrock 

Suspension:  spring  and  damper 

angle,  spacing 

Slope 

rates 

AASHO  curvature  safety  factor 

Surface  roughness 

Wheelbase  and  axle  positions 

Slope  stability  &  traction 

Area 

Tires:  section  height/width,  type. 

Throttle  setting 

Obstacles:  random  or  linear 

deflection/pressure 

On  &  off  road  visibility 

Obstacles:  height,  width,  length. 

Tracks:  road  wheels, 
sprockets/idlers,  track 

Surface:  dry,  wet,  icy 

angle,  spacing 

Drivetrain:  engine,  all  gearboxes. 

Vegetation:  tree  stem  size  and 

torque  converter 

Tire  deflection:  highway,  cross¬ 

spacing 

country  with/without  sand/snow 

Dual  tires 

Visibility 

Snow  chains 

Figure  4-5.  NRMM  partial  Scenario,  Terrain  and  Vehicle  data  requirements 


4.4  OUTPUT  FORMATS 

Predictions  file:  This  is  the  hackhone  of  the  NRMM  output  data  set.  It  provides  the  terrain  patch-hy-patch 
speed  and  limiting  factors  predictions.  For  each  unique  patch  of  terrain  it  predicts: 

•  The  tire  pressure/deflection  setting  that  offers  the  best  speed  (for  go  terrain). 

•  The  transmission  range  that  offers  the  best  speed  (for  go  terrain). 

•  The  OMNI  speed  for  the  patch  which  is  a  weighted  average  of  the  three  directions  of  travel  considered  (up, 
down  and  across  the  terrain). 

•  A  best  speed  prediction  for  each  of  the  three  directions  of  travel. 

•  A  limiting  reason  for  the  no-go  /  go  speed  predicted  for  each  of  the  three  directions  of  travel. 
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The  file  also  echoes  the  slope  and  size  of  the  patch  to  enable  filtering  and  post-processing  of  the  data;  for  more 
detailed  filtering  and  post-processing  the  patch  number  provides  a  common  key  back  to  the  terrain  data  file 
contents. 

The  data  in  this  file  can  be  aggregated  to  higher  level  forms  (e.g.  terrain  or  mission  type  summaries)  and  post- 
processed  in  more  detail  to  understand  platform  performance  envelopes  (e.g.  what  limits  performance  for 
specific  terrain  areas  or  speed  bands). 

Statistics  file:  This  file  contains  a  breakdown  of  the  limiting  reasons  associated  with  the  speed  and  no-go 
predictions  by  direction  of  travel.  It  also  contains  the  speed  curve  data  charts  presented  using  plain  ASCII 
characters  (as  a  hang-over  from  pre -Windows  days).  The  speed  curve  data  is  presented  in  both  percentile  and 
cumulative.  This  data  is  for  quick  reference,  it  is  not  intended  for  post-processing  into  other  forms. 

Cumulative  speeds  file:  Cumulative  speed  curves  are  the  standard  form  used  in  a  lot  of  analysis  reports  and 
quoted/referenced  in  requirements  documents. 


Example  truck  in  wet  conditions 
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30  40  50  60 

%  Access  of  totai  area 
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Worst/  siowest  going  %  of 
terrain 


Figure  4-6.  Example  cumulative  speed  curves 

In  effect  the  several  thousand  individual  predictions  are  put  into  descending  order  by  speed  and  presented  in 
speed  percentiles  (as  calculated  using  a  time  based  function).  The  chart  can  be  read  as  the  fastest  terrain  to  the 
left  of  the  horizontal  axis  and  the  slowest  to  the  right,  with  any  point  on  the  curve  giving  the  average  speed  for 
that  percentage  of  the  terrain. 
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Chapter  5  -  THEME  OVERVIEW 

As  stated  earlier,  ET-148  was  organized  around  seven  theme  areas.  The  goal  of  each  theme  is  the  following: 

•  Theme  1,  Requirements.  Capture,  consolidate,  and  summarize  desired  capabilities. 

•  Theme  2:  Methodologies.  Develop  a  plan  for  deriving  a  ground  vehicle  mohility  modeling  and  simulation 
architectural  specification  for  the  NG-NRMM. 

•  Theme  3:  Stochastics.  Descrihe  a  framework  for  a  stochastic  approach  for  vehicle  mohility  prediction  over 
large  regions  for  integration  into  a  NG-NRMM. 

•  Theme  4:  Intelligent  Vehicles.  Define  a  NG-NRMM  approach  and  requirements  for  mohility  assessment  for 
intelligent  vehicles. 

•  Theme  5:  Tool  choices.  Identify  critical  elements  for  a  physics-hased  next  generation  mohility  model 
utilizing  strengths  and  weakness  criteria  provided  hy  initial  “pros  and  cons”  review  of  current  NRMM. 
Identify  potential  solutions  throughout  the  technical  community  and  user  nations. 

•  Theme  6:  Input  Data  and  Output  Metrics.  Define  the  input/output  data  requirements  that  will  inform  the 
Next-Generation  NRMM  tool  development/selection  processes. 

•  Theme  7:  Validation  and  Verification.  Provide  a  process  for  conducting  a  successful  tool  and  software  code 
V&V  program  on  the  NG-NRMM. 

The  following  chapters  summarize  the  progress  made  hy  each  theme  toward  these  goals. 
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Chapter  6  -  THEME  1:  REQUIREMENTS 

Jody  Priddy  and  Michael  Bradbury 


6.1  GOALS  AND  DELIVERABLES 

Goals:  Capture,  consolidate,  and  summarize  key  mobility  modeling  capabilities  desired  by  the  team  member 
nations. 

Deliverable:  Documented  requirements  to  shape  AVT  recommendations. 

The  team  members  were  the  following: 


Country 

Name 

Canada 

Mayda,  William 

Czech  Republic 

Neumann,  Vlastimil 

UK 

Bradbury,  Michael:  Leader 

UK 

Suttie,  William 

USA 

Gunter,  David 

USA 

Jayakumar,  Paramsothy 

USA 

King,  Roger 

USA 

Letherwood,  Michael 

USA 

Priddy,  Jody:  Leader 

USA 

Shoop,  Sally 

6.2  INITIAL  SOLICITATION  OF  IDEAS 

During  the  first  teleconference  in  June  2014,  the  membership  was  asked  to  respond  to  three  questions: 

•  Things  you  like  about  NRMM 

•  Things  you  dislike  about  NRMM 

•  Prioritized  requirements  for  a  next-generation  NRMM 

Pages  and  pages  of  deliberative  responses  were  turned  in  by  those  members  of  the  team  that  were  major  users  of 
the  model.  The  complete  list  of  responses  is  included  in  Appendix  C. 
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The  long  list  of  responses  was  winnowed  down  and  divided  them  into  11  categories  of  requirements:  Output, 
Terrain,  Vehicles,  Human  Factors,  Modeling  and  Simulation  (M&S)  Methods,  Interfacing,  IT  Infrastructure, 
Software  Features,  Maintainability,  Expected  End  Users,  and  Distribution  Approach.  The  items  in  each 
category  are  included  below: 


Output 


•  Retain  NRMM-style  mobility  metrics  and  other  output  (e.g.,  off-road  speed,  %Nogo) 

•  Retain  strong  emphasis  on  comparative  mobility  analysis,  including  backwards  comparability  for  past 
NRMM  predictions 

•  Expand  mission  profile  definitions  (include  deformable  terrain  types) 

•  Establish  new  mobility  metrics  (e.g.,  compact,  user  friendly,  testable) 

•  Metrics  for  unmanned,  robotic,  perception,  and  sensor  system  performance 

•  Metrics  of  interest  to  all  NATO  partners 

•  Quantified  uncertainty  in  output  metrics 

•  Spatial  considerations  on  mobility  metrics  (e.g.,  inaccessible  “go”  islands) 

•  Generate  digital  maps  for  use  in  GIS  and  C2  tools 

•  Influence  of  potential  soil  moisture/strength  changes 

•  Performance  based  on  simulations/predictions  for  developmental  testing 

•  Powertrain  performance  (e.g.,  speed  on  slopes,  cooling  limits) 

•  Euel  economy  and  range,  efficiency 

•  3-D  vehicle  stability  metrics  (e.g.,  rollover,  lane  change,  steering  stability,  split  mu) 

•  Dynamic  stability  control  metrics  (e.g.,  for  ABS,  ESC  performance) 

•  Steering/tuming  performance  metrics 

•  Urban  maneuverability  metrics 

•  Improved  terrain  roughness  ride  quality  metrics  (including  asymmetric  terrain) 

•  Improved  linear  feature  obstacle  crossing  performance  metrics 

•  Swimming  and  fording  performance,  including  intrinsic  amphibious  characteristics 

•  Rut  depth,  including  multipass 
Terrain 

•  Increased  global  coverage 

•  Updated  terrain  data  sets 

•  Improved/expanded  terrain  definition  (e.g.,  scale-invariant  descriptions) 

•  Expand  terrain  profile  definitions  (e.g.,  specify  deformable  terrain  features) 

•  East  and  facile  methods  for  determining  theater-specific  terrain  characteristics 

•  Make  use  of  higher  resolution  terrain  data  sources  (e.g.,  LIDAR) 

•  Make  use  of  modern  GIS  terrain  data  sources 

•  Measurable  and  attainable  terrain  characteristics 

•  Comprehensive  terrain  features  and  range  of  characteristics 

•  Soil  characteristics,  including  various  strength  parameters  for  alternative  terramechanics  approaches 
(e.g.,  RCI,  internal  friction,  cohesion) 

•  Potential  variations  in  soil  moisture/strength 

•  Snow  characteristics  (e.g.,  depth) 

•  Ereeze/thaw  soil  conditions 

•  Road  characteristics 
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•  Split  mu  features  (e.g.,  gravel  shoulder,  road  edge) 

•  Urban  features 

•  Terrain  roughness,  including  asymmetry  features 

•  Improved  roughness  metrics  (better  than  RMS,  stationary,  ergodic,  spectrally  general) 

•  Rocky  terrain  features  (e.g.,  rocky  shore  in  surfzone) 

•  3-D  linear  feature  obstacles  (e.g.,  gaps,  barriers) 

•  Library  of  selectable  and  expandable  standard  obstacles 

•  New  standardized  obstacle  types  (e.g.,  rubble  pile,  embedded  hard  obstacles  in  deformable  terrain) 

•  3-D  water  feature  obstacles  (e.g.,  streams,  ponds,  lakes,  rivers,  oceans,  surfzones,  ship  launch) 

Vehicles 

•  Robust  comprehensive  vehicle  characteristics 

•  Attainable  vehicle  characteristics 

•  Multi-fidelity  from  simple  to  rigorous  characterizations 

•  Modern  suspensions  (e.g.,  independent,  active,  semi-active) 

•  Modern  braking  systems  (e.g.,  ABS) 

•  Modern  powertrain  systems  (e.g.,  TCS,  ESC,  ABM,  hybrid,  electric) 

•  Powertrain  cooling  systems 

•  Computer  controllers  (e.g.,  ABS,  TCS,  ESC,  ABM,  active/semi-active  suspensions) 

•  Steering  systems  (e.g.,  skid  steering) 

•  Pneumatic  tires  (e.g.,  bias  ply,  radial) 

•  Tracks  (e.g.,  flexible  steel  link,  rubber  band) 

•  Non-pneumatic  wheels  (e.g.,  rigid,  airless) 

•  Size  and  weights  including  smalPlight  robots  to  large/heavy  main  battle  tanks 

•  Unmanned,  robotic,  perception,  and  sensing  systems 

•  Undercarriage  clearance  geometry 

•  Intrinsic  amphibious  characteristics  (e.g.,  buoyancy) 

Human  Eactors 

•  Human  tolerance  limits  over  rough  terrain  (including  asymmetric  terrain) 

M&S  Methods 

•  Include  multi-fidelity  modeling  options  from  simple  to  rigorous,  empirical  to  physics  based 

•  Improved  tire/track-soil  interface  modeling 

•  3-D  tire/track  models 

•  3-D  physics  based  models  of  deformable  terrain  (e.g.,  soil,  snow) 

•  Include  alternative  terramechanics  approaches 

•  Include  physics  based  dynamic  simulations 

•  3-D  MBD  for  vehicle  dynamics,  including  rigid  and  flexible  bodies 

•  Methods  for  quantifying  powertrain  and  braking  torque  delivered  to  each  traction  element  (e.g.,  wheels, 
tracks) 

•  Include  dynamic  simulation  of  powertrain  and  braking  performance 

•  Driver  models  for  simulation  control 

•  Uncertainty  quantification  (e.g.,  Monte  Carlo  simulation) 

•  Design  of  experiments  methods 

•  Include  response  surface  methods  or  other  approximation  methods 

•  Chassis/undercarriage  collision  and  resistance  methods 

•  Methods  for  dynamic  simulation  of  amphibious  operations  (e.g.,  CED) 
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•  Methods  for  sensor,  perception,  and  autonomy  system  modeling 
Interfacing 

•  Interfacing  with  existing  GIS  tools  (input  and  output) 

•  Interfacing  with  existing  3-D  MBD  tools 

•  Driver  feedhack  loop  for  speed  control  (e.g.,  controller  HITL) 

IT  Infrastructure 

•  Enable  use  of  modem  HPC  resources 

•  Maintain  portability  and  desktop  computing  capability 
Software  Features 

•  Modem  software 

•  Easy  to  install 

•  User  friendly 

•  Modular  software  architecture 

•  Good  error  handling 

•  Runs  quickly  (e.g.,  single  run  in  minutes  or  less,  not  hours  or  days) 

•  Enhanced  user  interface  for  inputs,  outputs,  and  data  management  (e.g.,  GUI) 

•  Enhanced  graphical  output  (e.g.,  graphs,  charts,  visuals) 

•  Include  different  versions  or  user  modes,  from  "lite"  to  "expert" 

•  Include  input  and  output  compatible  with  common  existing  analysis  tools  (e.g.,  MATLAB, 
spreadsheets,  GIS  tools) 

•  Ability  for  plug-ins,  add-on  modules  (e.g.,  alternate  terramechanics  modules,  controller-logic  modules) 

•  Provide  multi-fidelity  analysis  options,  with  associated  input  data  requirements  ranging  from 
simple/limited  to  robust/extensive 

•  Allow  easy  variation  of  select  parameters  for  quick  "what  if  scenarios  by  non-specialists  end-users 
(e.g.,  weight,  power,  number  of  axles) 

•  Provide  clear,  robust  diagnostics  and  detail  options  (e.g.,  nogo  reasons  to  include  multiple  reasons, 
access  to  intermediate  and  lower  level  results) 

•  Include  library  of  terrain  features  that  are  selectable  and  tailorable  to  vehicle  and  mission  requirements 
(e.g.,  obstacles) 

•  Allow  terramechanics  changes,  alternatives,  and  comparisons 
Maintainability 

•  Need  formal  mechanism  for  software  maintenance 
Expected  End  Users 

•  NATO  community 

•  Non-specialists  end  users 

•  Expert  end  users 
Distribution  Approach 

•  Improved  distribution  with  NATO  accessibility 

•  Could  include  commercial,  open  source,  or  both 

•  Available  and  supported  for  use  by  industry 

•  Prefer  minimal  licensing/maintenance  costs  for  use  in  government  purposes 

6.3  THE  USER 

When  setting  requirements  it  is  also  necessary  to  understand  the  needs  and  expectations  across  the  stakeholder 
community.  For  the  purposes  of  Next  Generation  NRMM,  the  User  is  considered  to  be  the  software  operator. 
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Four  broad  categories  of  User  have  been  identified  as  follows: 

•  Supervised  practitioner:  Someone  who  will  require  support  and  guidance;  assistance  with  some  aspects  of 
data  input,  configuration,  running  the  model,  post-processing  and/or  presenting  the  resulting  analysis  to  the 
Customer. 

•  Practitioner:  Someone  that  can  interpret  the  Customers’  needs,  then  define  and  execute  analysis  that 
provides  appropriate  decision  support  without  supervision  or  guidance.  Someone  that  can  adapt  how  the 
software  is  used  if  needed  but  may  require  advice  regarding  the  execution  or  validity  of  that  adaptation. 

•  Expert  User:  Somebody  who  not  only  is  proficient  in  utilising  the  software  to  provide  decision  support  but 
understands  the  science  behind  it  and  the  underlying  functionality.  This  person  is  a  recognised  authority  on 
the  subject  and  can  truly  attest  as  to  whether  the  software  is  being  used  in  a  viable  and  reliable  manner. 

•  Operational  planner:  This  person  has  to  operate  independently,  likely  remotely  from  the  core  community, 
relying  largely  on  re-using  data  (e.g.  vehicle  and/or  terrain  files)  for  typical,  well  understood  analysis  tasks, 
reaching  back  to  core  community  practitioners  as  needed. 

The  initial  requirements  identified  in  this  report  do  not  discriminate  between  these  User  types.  As  requirements 

develop  into  formal  User  and  System  requirements  documents  or  a  technical  specification  they  can  be  used  to 

describe,  qualify  and  differentiate  functionality  as  needed. 


6.4  KEY  NEW  REQUIREMENTS 

The  theme  membership  took  this  the  requirements  from  Section  6.2  and  further  consolidated  them  into  fewer 
categories.  New,  or  enhanced,  requirements  have  been  identified  across  four  categories: 

•  System:  Platform  types  within  scope. 

•  Modeling:  Technologies  and  subsystems  within  scope. 

•  Analysis:  Problem  spaces  or  analysis  questions  within  scope. 

•  Output:  Metrics,  results  formats  and  exploitation  interfaces  within  scope. 


The  final  list  of  key  new  requirements  for  a  Next-Generation  NRMM  model  was  separated  into  Near-Term 
Priorities  (Threshold)  and  Far-Term  Priorities  (Objectives)  as  shown  in  Figure  6.1.  Note  that  when  an  item 
appears  in  both  near  and  far-term,  it  is  in  recognition  that  either  ground  work  is  needed  now  to  enable  far-term 
priorities  or  where  a  lesser  solution  is  feasible  as  a  step  along  the  development  path.  Also,  although  a  GUI  and 
animation  are  not  explicitly  stated  as  Key  New  Requirements,  they  are  desirable  in  current  and  future  software 
options. 

Vehicles  may  be  manned  or  unmanned,  in  either  case  human  control  may  be  supplemented  by  varying  levels  of 
autonomy  to  assist  or  replace  (for  periods  of  time)  the  operator.  From  the  perspective  of  mobility  modeling  this 
has  implications  from  the  terrain  data  definition  to  the  modeling  strategy  (e.g.  driver  prudence/constraints).  The 
use  of  the  term  'autonomous  vehicles'  within  this  report  is  within  that  context.  See  Chapter  9  on  Intelligent 
Vehicles  for  more  information. 
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Category 

Sub-category 

Near-Term  Priorities  for 

NG-NRMM  Threshold 

Far-Term  Priorities  for 

NG-NRMM  Objective 

Vehicle  Type 

Wheeled,  tracked,  autonomous 

Legged,  autonomous 

New  System 
Capabilities 

Vehicle  Scale 

Conventional  manned  vehicles 

Lighter  and  smaller  vehicles 

Terrain  Scale 

Rz^ional,  varied  resolutions 

Global,  v-ri  jd  rocoiutions 

Suspension  Types 

Passive,  semi-active,  active 

Active 

Control  Types 

Driver,  ABS,  TCS,  ESC,  ABM,  CTIS, 

autonomy 

Autonomy 

New  Modeling 
Capabilities 

Sub-systems 

Steering,  powertrain,  autonomy 

Autonomy,  human  cognition 

Model  Features 

3D  Physics  based  models 

Multibody  dynamic  vehicle  models 
Flexible  body  models 

Detailed  tire  and  track  models 

Terrain  models  (e.g.  Bekker-Wong) 

Terrain  models  (e.g.  DEM,  FEM) 
Stochastic  models 

User  Type 

Analyst/Expert 

Operational  Planner 

Environment  Types 

On-road,  off-road 

Urban,  soil,  snow/ice 

Urban 

New  Analysis 
Capabilities 

Powertrain  Performance 

Grading,  turning,  fuel  economy 

Cooling 

Amphibious  Operations 

Fording,  swimming 

Computations 

Efficiency  -  fidelity  trade  off 

Fligh  fidelity 

High  performance 

New  Output 
Capabilities 

Assessment  Types 

Mobility  performcnc.  ii. 
operational  context 

Metric  Considerations 

Verifiable  mobility  metrics 

Figure  6-1.  Key  New  Requirements  for  Threshold  and  Objective  NG-NRMM.  The  colors  indicate  gap 
areas  in  Mobility  Mapping  (Light  Blue),  Environmental  Modeling  (Green),  Intelligent  Vehicle  (Red), 
Stochastics  (Purple),  Computational  Performance  (brown)  and  Verification  and  Validation  (Dark  Blue). 
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6.5 


NEXT  STEPS 


Theme  1  has  highlighted  Key  New  Requirements  which  address  both  capability  sustainment  (more  accurately 
restoration)  and  growth.  In  essence  there  are  two  logical  next  steps: 

Requirements  documents:  Turn  the  Key  New  Requirements  into  User  and  System  requirements  (or  some 
other  form  of  technical  specification)  with  Specific,  Measurable,  Achievable,  Relevant  and  Time -bound 
(SMART)  requirements.  Given  it  is  unlikely  a  single  solution  will  meets  all  requirements  it  is  essential  to  the 
collaborative  effort  that  priorities  are  agreed  within  these  requirements  so  that  collectively  requirements  can  be 
traded  or  risk  taken  against  them. 

Requirements  documents  are  needed  to  ensure  the  Next  Generation  NRMM  delivers  the  right  capability  and  that 
the  community  best  appreciates  the  effort  and  risks  therein.  Detailed  requirements  documents  will  be  key  to 
securing  national/intemational  funding  and  support  from  academia/industry  in  addition  to  any 
commercial/contractual  arrangements  with  suppliers. 

Requirements  roadmap:  Generate  a  requirements  roadmap  in  parallel  (to  refining  requirements)  defining  the 
relationships  and  dependencies  between  the  requirements.  E.g.  you  cannot  perform  data  fusion  across  all  terrain 
types  until  you  can  model  all  terrain  types. 

Example: 

•  Current  NRMM  looks  at  on  and  off-road  predictions  in  isolation. 

•  To  provide  effective  decision  support  with  a  growth  path  to  Operations,  Next  Generation  NRMM  needs  to 
consider  data  and  analysis  fusion  across  the  on/off  roads  terrain  types. 

•  Eurther,  at  a  minimum  it  must  consider  the  interface  with  urban  landscapes,  if  not  the  assimilation  of.  To  do 
so,  it  must  have  an  urban  mobility  definition  or  assessment  capability. 

•  As  this  new  capability  looks  at  the  fused  terrain  with  greater  fidelity  it  will  need  to  consider  directionality  in 
context  (i.e.  actual  as  opposed  maximum  slope)  and  uncertainty  (stochastics). 

This  is  needed  to  allow  for  effective  programme  management  and  delivery  for  Next  Generation  NRMM. 

In  summary,  while  the  current  level  of  requirements  definition  is  sufficient  for  the  community  to  progress 
toward  improved  simulation  and  prediction  accuracy,  it  is  insufficient  for  program  delivery.  To  finalise  the 
requirements  there  is  a  dependency  on  the  other  themes,  which  in  turn  is  in  practical  terms  dependent  on 
currently  available  software  solutions  and  their  potential  growth  paths.  The  ultimate  exploitation  of  a  well- 
defined  requirement  beyond  programme  delivery  could  be  the  building  blocks  for  the  definition  of  a  mobility 
modeling  standard. 
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Chapter  7  -  THEME  2:  METHODOLOGY 

Michael  McCullough 


7.1  GOALS  AND  DELIVERABLES 

Through  the  course  of  the  ET,  a  Methodology  Development  Vision  was  proposed  for  four  different  levels  of 
model  complexity.  As  shown  in  Figure  1  helow,  the  current  model,  the  NRMM  standard  release,  is  empirical. 
The  Exploratory  Team  considered  three  levels  of  complexity  for  the  Next-Generation  NRMM  as  shown  in  the 
last  three  columns,  an  Enhanced  Empirical  Model,  a  Semi-Analytical  and  an  Analytical.  The  decision  was  that 
the  Methodology  would  he  to  develop  the  Open-Architecture  type  models  with  a  Semi-Analytical  being  most 
possible  in  this  time  frame  but  with  future  efforts  aimed  toward  an  analytical  model. 


Model 

Component 

Model  Accuracy  and  Resolution 

Current 

r^iirront  _ 

Open  Architecture  Model 

Enhanced 

Near-Term 

Long-Term 

Mobility 

Mapping 

NRMM 

Operational 

Module 

0) 

NRMM 

Operational 

Module 

V) 

Modified 

NRMM 

Operational 

Module 

Modular,  Expandable, 
Documented,  Verified, 
Mobility  Mapper  with  Long 
Term  NATO  CM  support 

Off-Road 

Mobility 

o 

1  NRMM  1  NRMM*  «Bekker/Wong 

®  Se-rramechanics 

FEM  IDEM 

Vehicle 

Dynamics 

■E  ^  o  .. 

.^HDYN  (2D)  «  3DMBD  ^ 

%  £  1 

U) 

2  Integrated  Deformable 

S  dynamic  terrain 

O 

Intelligent 

Vehicle 

P  ^  2;losed  loop  3D 

^  ons  an  g  ^^jabie  speed  h  path  following 

^  with  sensors 

Z 

Al  with  analytical  sensor- 
terrain  interaction  in 
feature-rich  environments 

Compute 

Platform 

Desktop 

Desktop 

’.  ulti-Threaded 
Desktop 

HPC 

Figure  7-1.  Next-Generation  NRMM  Methodoiogy  Deveiopment  Vision 


“Open  Architecture  model”  refers  to  an  enduring  non-preferential  realization  of  the  model  that  is  implemented  at 
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a  higher  level  of  ahstraction  that  will  he  inclusive  of  a  variety  of  specific  executable  implementation 
environments,  all  validated  legacy  models  and  input  data,  while  also  establishing  a  framework  for  future 
innovation.  It  was  proposed  and  accepted  that  the  simplest  form  of  this  higher  level  of  abstraction  is  a  set  of 
mobility  model  standards  and/or  specifications.  Thus,  the  acronym  NORMMS  was  coined  for  NATO 
Operational  Reference  Mobility  Modeling  Standards.  The  NORMMS  framework  was  defined  as  a  ground 
vehicle  mobility  modeling  and  simulation  architectural  specification  applicable  to  the  full  range  of  ground 
vehicle  geometric  scales  that  promotes  standardization,  integration,  modular  interoperability,  portability, 
expansion,  verification  and  validation  of  vehicle -terrain  interaction  models  at  multiple  levels  of  theoretical  and 
numerical  resolution  for  use  in  vehicle  design,  acquisition  and  operational  mobility  planning. 

The  Methodology  team  members  are  shown  in  Figure  2.  A  variety  of  points  of  view  were  expressed  and  written 
drafts  of  specific  proposed  standards  were  developed  by  some  of  the  team  members  which  provide  examples  of 
specific  issues  and  the  level  of  detail  required  in  the  NORMMS  specification  statements.  Appendix  D  contains 
the  text  of  these  examples.  The  team  also  developed  the  following  high  level  goals: 

•  Develop  a  plan  for  deriving  a  ground  vehicle  mobility  modeling  and  simulation  architectural 
specification,  or  NORMMS,  defining  the  content  of  the  Next  Generation  NRMM. 

•  Leverage  the  capabilities  of  team  members 

•  Address  all  Requirements  from  Theme  1 

•  Integrate/coordinate  with  methods  work  done  by  Themes  3-7 

The  theme  members  are  listed  below: 


Country 

Name 

Canada 

Wong,  J.Y. 

Czech  Republic 

Rybansky,  Marian 

Denmark 

Balling,  Ole 

Germany 

Gericke,  Rainer 

Poland 

Glowka,  Jakub 

Poland 

Wrona,  Jozef 

USA 

Gunter,  David 

USA 

Hodges,  Henry 

USA 

lagnemma,  Karl 

USA 

Jain,  Abhi 

USA 

Jayakumar,  Paramsothy 

USA 

Letherwood,  Michael 

USA 

McCullough,  Michael:  Leader 

USA 

Ngan,  James 

USA 

Priddy,  Jody 

USA 

Ward,  Derek 

USA 

Wojtysiak,  Brian 
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The  Theme  members  developed  a  draft  NORMMS  specification  for  both  the  Semi-Analytical  (Threshold 
capability)  and  the  Analytical  (Objective)  Capability,  starting  with  the  high  level  summary  given  previously  in 
Figure  7-1.  The  detailed  requirement  set  forms  the  basis  for  measuring  progress  and  completeness.  Because  it  is 
impossible  to  predict  all  possible  mobility  metrics  and  these  may  change  with  every  application,  the  open 
architecture  is  necessary  to  accommodate  the  required  flexibility  being  expressed  by  the  ultimate  end-users. 
Figure  7-2  depicts  schematically,  and  with  the  color  scheme,  the  flow  of  input/output  requirements  that  are 
expected  to  be  typical  for  future  applications  of  the  NG-NRMM.  This  figure  depicts  a  significant  mobility 
mapping  effort  (Theme  6)  that  can  be  decoupled  at  the  executable  level  from  the  vehicle  terrain  interaction 
(VTI)  modeling  perspective.  Mobility  mapping  tools  that  allow  operations  and  overlays  with  GIS  and  remotely 
sensed  data  are  currently  being  used  for  this  purpose  and  provide  a  ready  suggested  tool  set  for  the  NG-NRMM 
mobility  mapping  component  that  allows  mobility  to  be  assessed  at  more  global  levels. 

VTI  modeling  is  its  own  focus  area  and  is  driven  by  the  end-use  needs  of  the  vehicle  design,  acquisition  and/or 
operational  mobility  planning  communities.  These  driving  requirements  are  frequently  requested  as  map 
enabled  mobility  metrics,  but  just  as  often  are  summary  level  performance  metrics  reduced  to  averages  across 
specific  regions  of  terrain  and  scenario  combinations,  and  are  therefore  not  required  to  be  mapped.  The 
additional  terrain  data  requirements  and  higher  levels  of  resolution  for  detailed  VTI  simulations  are  one  of  the 
core  research  and  development  issues  distinguishing  the  current  NRMM  from  the  next  generations  envisioned 
by  this  FT,  i.e.,  the  semi-analytical  and  analytical.  This  additional  and  higher  resolution  terrain  data  is  used  in 
the  local  mobility  models.  On  the  lower  end  of  the  chart,  the  computer  aided  engineering  software  and  computer 
hardware  spectrums  are  currently  decoupled  at  the  executable  level  because  the  general  purpose  vehicle 
modeling  codes  are  ported  to  all  hardware  platforms,  but  for  detailed  deformable  terrain  models  employing 
continuum  models  that  take  advantage  of  physics  co-processers,  or  general  purpose  graphics  processing  units 
(GPGPUs),  there  will  be  a  tighter  coupling  between  the  software  and  hardware.  Current  state  of  the  practice  and 
successful  use  of  VTI  models  has  identified  multi-body  dynamics  software  as  the  primary  modeling 
environment  that  is  readily  available,  significantly  validated  across  a  practically  limitless  range  of  vehicle 
morphologies,  and  meets  the  goals  and  requirements  for  the  integration  of  all  of  the  desired  capabilities 
identified  for  both  threshold  and  objective  NG-NRMM. 
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Figure  7-2:  Next  Generation  NRMM  Schematic  and  NORMMS  Requirements  Flow 

The  light  blue  hox  in  Figure  7-2  is  the  M&S  integrating  environment  (MSIE).  MISE  presents  a  unique 
opportunity  to  identify  a  modeling  process  integration  tool  that  enables  the  envisioned  open  architecture  for 
NG-NRMM  through  the  implementation  of  executable  NORMMS.  The  MSIE  tool  would  enable  the  Research 
Technical  Group  to  capture  decisions  about  algorithms  and  metrics,  and  simultaneously  implement  them  in  a 
form  that  is  ultimately  executable,  portable,  enduring,  and  promotes  easy  collaboration  and  distribution  of  the 
standard  algorithms  with  non-preferential  interfaces  to  the  simulation  codes  and  GIS  tools  that  are  already  seen 
as  essential  components  of  NG-NRMM.  A  key  requirement  of  the  MSIE  is  the  ability  to  construct 
customizable  templates  that  support  integration  of  the  wide  variety  of  multibody  dynamics,  multiphysics,  and 
GIS  tools  that  have  become  indigenous  to  the  various  organizations  and  countries  with  stakeholder  interest  in  the 
Next  Generation  NRMM.  By  way  of  example,  a  potential  candidate  for  this  MSIE  might  be  the  Windows/DOS 
command  environment  combined  with  EXCEL  and  Visual  Basic  or  Visual  Studio.  However,  there  may  be  more 
modern  tools  such  as  Python  which  are  ultimately  more  enduring  and  directly  align  with,  and  achieve,  the  RTG 
goals  for  NG-NRMM.  There  are  also  commercial  tools  associated  with  Computer  Aided  Engineering  (CAE) 
which  share  the  same  vision  such  as  SimManager/SimExpert  from  MSC  and  COMET.  The  RTG  could  choose 
to  adopt  one  of  these  as  well,  although  they  would  require  that  financial  barriers  to  entry/ownership  be  small  and 
must  demonstrate  an  enduring  path  to  the  future. 

It  should  be  noted  in  the  context  of  the  high  level  mobility  metrics  that  the  current  version  of  the  NRMM 
Operational  Module  provides  a  valuable  starting  point.  It  is  written  in  EORTRAN  and  can  be  adopted  in  parts  or 
even  translated  into  the  new  MSIE  environment  language.  This  is  considered  a  valuable  first  step  for  the  RTG 
after  a  decision  on  the  MSIE  is  made.  Based  on  this  observation  the  current  NRMM  mobility  “reason  codes”  are 
therefore  considered  a  valuable  starting  list  of  NORMMS  attributes. 
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7.2.  DRAFT  NORMMS  SPECIFICATION 

1.  New  System  Capabilities.  NG-NRMM  shall  be  implemented  in  vehicle  modeling  environments  that 
have  system  modeling  capabilities  supporting  template  based  construction  of  a  broad  range  of  tracked, 
wheeled,  and  autonomous  vehicles  at  the  scale  of  conventional  manned  vehicles  interacting  with  terrain 
data  sets  at  the  local  and  regional  levels  of  resolution  (threshold)  with  future  expansion  capabilities  to 
include  all  vehicle  morphologies,  levels  of  autonomy,  and  terrain  interactive  capabilities  extending  to 
global  data  sets. 

2.  New  Modeling  Capabilities.  The  threshold  NG-NRMM  shall  be  implemented  in  vehicle  modeling 
environments  that  have  subsystem  modeling  capabilities  to  include:  3-D  physics  based  multi-body 
models,  inclusive  of  flexible  bodies,  passive,  semi-active  and  active  control  systems  (e.g.  ABS,  TCS, 
ESC,  ABM,  CTIS),  human  driver  models,  autonomous  control,  detailed  powertrain  models,  detailed 
tire  and  track  models  interacting  with  deformable  terrain  models  based  on  semi-analytical  terrain 
response  models  equivalent  in  complexity  to  Bekker-Wong  pressure  sinkage  models  and  Janosi  type 
shear  response  models.  The  objective  NG-NRMM  shall  be  implemented  in  vehicle  modeling 
environments  that  are  fully  inclusive  of  emerging  advanced  deformable  terrain  modeling  methods  such 
as  DEM,  SPH,  and  DVI  as  well  as  advanced  autonomy  models  including  human  cognition  and  the 
associated  terrain  descriptive  and  interactive  simulation  capabilities  required  to  support  those.  The 
objective  NG-NRMM  shall  also  include  proven  and  accepted  methods  for  analyzing  and  accounting  for 
the  primary  stochastic  attributes  of  mobility  modeling. 

3.  New  Analysis  Capabilities.  The  threshold  NG-NRMM  shall  be  implemented  in  vehicle  modeling 
environments  that  have  expanded  analysis  tools  consistent  with  the  Eigure  7-1  attributes  for 
Environment  Types,  Powertrain  Performance,  Amphibious  Operations,  and  Computational 
architectures.  Objective  NG-NRMM  shall  be  implemented  in  modeling  environments  permitting 
automated  methods  of  interacting  with  urban  terrains  and  taking  advantage  of  massively  parallel 
computers  and  physics  co-processers. 

4.  New  Output  Capabilities.  The  NG-NRMM  shall  implement  all  output  data  required  by  advanced 
applications  of  mobility  data  at  the  operational  level  including  the  ability  to  rapidly  compute  new  and 
unusual  mobility  metrics  that  are  verifiable  and  map  enabled  using  GIS  based  visualization  tools. 

7.3.  DETAILED  NORMMS  COMPLIANCE  ASSESSMENT 

The  draft  specification  above  is  intended  to  form  the  high  level  framework  from  which  a  fully  detailed 
specification  can  be  developed  that  permits  any  interested  and  NATO  authorized  organization  to  develop  a  NG- 
NRMM  that  accomplishes  the  goals  of  this  effort.  In  cases  of  existing  capabilities,  this  exercise  may  simply  be  a 
process  of  verification  of  compliance  to  the  updated  expectations  of  NG-NRMM  at  the  threshold  or  objective 
levels,  respectively.  In  the  immediate  shorter  term,  the  NORMMS  development  process  can  also  become  the 
broader  context  within  which  the  contributions  of  the  other  themes  of  this  ET  are  captured  and  adopted. 

Eor  the  future,  the  draft  NORMMS  specification  is  also  intended  to  be  a  living  document  that  can  be  further 
developed  to  higher  levels  of  resolution  and  detail  as  necessary  to  accomplish  the  on-going  goals  of  the  NG- 
NRMM  development  process.  Early  in  that  future  process,  they  will  provide  a  checklist  of  requirements 
against  which  proposed  modeling  environments  can  be  assessed  with  respect  to  their  potential  to  implement  a 
NG-NRMM  capability.  Later,  with  further  detailed  elaboration,  this  can  evolve  into  a  verification  and 
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validation  dashboard  used  to  accredit  a  given  proposed  capability.  Finally,  if  the  NATO  development  team 
comes  to  this  conclusion,  it  can  be  used  as  a  basis  for  a  specification  of  a  NATO  sponsored  software 
capability  that  implements,  in  part,  or  in  whole,  a  common  NATO-owned  and  distributed  NG-NRMM 
implementation.  An  example  of  such  a  progress  measurement  dashboard  is  shown  in  Figure  7-3  below. 
Development  and  ratification  of  the  precise  entries  representing  the  desired  attributes  for  a  NORMMS 
description  of  the  NG-NRMM  is  an  early  goal  of  the  RTG  effort. 
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Figure  7-3:  Example  Progress  Measurement  Dashboard  for  development  of  a  NORMMS  specification. 
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Chapter  8  -  THEME  3:  STOCHASTICS 

Karl  lagnemma  and  Ramon  Gonzalez 


8.1  GOALS 


The  objective  of  the  proposed  research  is  to  describe  a  framework  for  a  stochastic  approach  for  vehicle 
mobility  prediction  over  large  regions,  for  integration  into  a  NG-NRMM. 

The  team  members  are  shown  below: 
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8.2  INTRODUCTION 

It  is  well-known  that  before  attempting  a  mission  involving  a  ground  vehicle  in  off-road  conditions  a  reliable 
and  comprehensive  analysis  of  the  mobility  capabilities  of  such  a  vehicle  is  desired.  This  goal  can  be  solved 
by  means  of  computer  simulation,  where  both  terrain  profile  and  vehicle -terrain  interaction  play  a  key  role. 
Traditionally,  this  analysis  considers  nominal  values  for  the  key  variables  involved  in  the  simulation.  This 
leads  to  unreliable  and  limited  results  due  to  the  uncertainty  present  in  those  variables.  Key  variables  include 
those  related  to  terrain  geometry  and  terrain  physical  properties.  Vehicle  parameters  and  their  dependencies 
should  also  be  addressed  for  a  full  stochastics  treatment,  but  were  not  considered  here. 

Terrain  geometry  information  typically  comes  from  remote  sensory  sources  (i.e.  radar  technology,  imagery 
methods,  etc.).  Those  techniques  lead  to  models  of  the  terrain  with  uncertainty  associated  with  the  spatial 
position  of  data  points.  Thus,  any  elevation  model  of  the  terrain  is  corrupted  by  uncertainty.  Digital  Elevation 
Models  (DEMs)  produced  by  the  US  Geological  Survey  agency  are  a  good  example  of  this  issue. 

Spatial  variability  of  physical  terrain  properties  (e.g.  soil  cohesion  and  internal  friction  angle)  also  leads  to 
uncertainty  in  vehicle-terrain  interaction  models.  In  addition,  measurement  methods  of  the  soil  properties  are 
uncertain  in  nature. 

Here,  we  describe  a  framework  for  a  stochastic  approach  for  vehicle  mobility  prediction  over  large  regions  (> 
5x5  [Km^]).  This  method  could  form  part  of  a  Next  Generation  NRMM  tool.  In  this  framework,  a  model  of 
the  terrain  is  created  using  geostatistical  methods.  The  performance  of  a  vehicle  is  then  evaluated  while 
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considering  the  terrain  profile  and  the  vehicle-terrain  interaction.  In  order  to  account  for  uncertainty,  Monte 
Carlo  simulations  are  performed,  leading  to  a  statistical  analysis.  Uncertainty  in  elevation  is  due  to  the  new 
interpolated  terrain  model  to  a  higher  spatial  resolution  than  the  original  DEM  (through  a  geostatistical 
method  called  Ordinary  Kriging).  On  the  other  hand,  uncertainty  in  soil  properties  is  obtained  considering  the 
variability  of  the  parameters  involved  in  the  well-known  Bekker-Wong  (BW)  model  [Bekker,  1969;  Wong, 
2001]. 

8.3  IDENTIFICATION  OF  NEEDS  AND  CHALLENGES 

After  a  review  of  the  current  (deterministic)  NRMM  [Rula  and  Nuttall,  1971;  Haley  et  ah,  1979]  and  the 
suggestions  proposed  to  date  to  formulate  a  stochastic  NRMM  [Lessem  et  ah,  1992;  Lessem  et  ah,  1993],  the 
following  needs  and  challenges  have  been  identified: 

•  Previous  attempts  to  convert  NRMM  from  a  deterministic  framework  to  a  stochastic  one 
have  failed  in  the  core  component  of  a  stochastic  procedure,  that  is,  the  origin  of  uncertainty. 
No  formal  mathematical  reasoning  about  the  uncertainty  introduced  in  the  simulations  is 
given  in  [Lessem  et  al.,  1992;  Lessem  et  al.,  1993;  Lessem  et  al.,  1996]. 

•  An  efficient  numerical  solution  is  highly  recommended  in  Lessem’s  works.  So  far,  the 
proposed  (stochastic)  implementation  of  NRMM  requires  supercomputers  and  requires 
extensive  time  to  obtain  a  solution. 

•  Development  of  an  architecture  that  is  flexible  enough  to  accept  a  variety  of  information 
sources  is  required.  In  particular,  it  is  desired  to  be  able  to  use  standard  cartographical  models 
available  today,  that  is,  digital  elevation  models.  Worldwide  maps  are  freely  available  from 
the  US  Geological  Survey  agency  at  different  spatial  resolutions. 

•  The  output  of  the  current  NRMM  is  given  in  terms  of  a  deterministic  mobility  map.  This  map 
shows  the  average  cross-country  speed  between  two  points  in  a  given  region  for  a  given 
vehicle.  As  recommended  by  [Lessem  et  al.,  1992],  a  stochastic  analysis  should  be  given  in 
terms  of  probability  densities  rather  than  the  ranges  in  the  variables. 

•  The  current  NRMM  does  not  support  autonomous  mobility  (this  issue  was  pointed  out  in 
[Vong  et  ah,  1999]).  Notice  that  this  capability  is  highly  advisable  in  the  Next  Generation 
NRMM  because  current  and  future  defense  forces  include  autonomous  systems. 

8.4  RELATED  WORK 

This  section  summarizes  the  main  publications  framed  in  the  context  of  this  work.  Firstly,  the  state-of-the-art 
in  the  field  of  mobility  prediction  is  analyzed.  After  that,  a  study  of  the  literature  related  to  Geostatistics  is 
presented.  Finally,  a  review  of  the  previous  research  framed  in  the  context  of  stochastic  NRMM  is  included. 


8.4.1.  Mobility  prediction 

•  Many  publications  cope  with  3D  path  planning  in  the  close  vicinity  of  an  autonomous  mobile  robot 
[Goldberg  et  al.,  2002;  Norouzi  et  al.,  2012;  Trease  et  al.,  2011].  Those  approaches  are  generally  not 
appropriate  for  planning  longer  routes  over  large  environments  because  they  are  based  on  sensors  that 
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perceive  only  the  surrounding  environment  (e.g.  stereovision,  LIDAR). 

•  An  important  research  effort  has  been  made  in  the  field  of  comhining  remote  sensor  and  ground 
sensor  data  [Helmick  et  ah,  2009;  Stentz  et  ah,  2002;  Vandapel  et  ah,  2006].  The  solutions  addressed 
in  this  report  are  in  fact  inspired  hy  those  papers. 

•  Mobility  prediction  has  also  been  considered  in  terms  of  the  vehicle -terrain  interactions  [Ishigami  et 
ah,  2009;  Willoughby  et  ah,  2006].  For  example,  in  [Ishigami  et  ah,  2009],  a  statistical  method  for 
mobility  prediction  considering  uncertainty  in  terrain  physical  properties  (soil  cohesion  and  internal 
friction  angle)  is  proposed. 

•  Uncertainty  in  control  actions  is  also  taken  into  account  in  the  literature.  For  instance,  in  [Peynot  et 
ah,  2014],  the  authors  define  mobility  prediction  as  the  problem  of  estimating  the  likely  behavior  of  a 
planetary  exploration  rover  in  response  to  given  control  actions  on  a  given  terrain. 

•  Some  research  projects  focus  on  the  reconstruction  of  a  3D  surface  from  sparse  data  obtained  from  a 
remote  sensor  [Hadsell  et  ah,  2009;  Kweon  and  Kanade,  1992].  However,  these  works  do  not 
consider  the  second  goal  of  this  research,  that  is,  stochastic  mobility  prediction. 

8.4.2.  Geostatistics 

Geostatistics  aims  at  providing  quantitative  descriptions  of  natural  variables  distributed  in  space -time  [Chiles 
and  Delfiner,  2012;  Isaaks  and  Srivastava,  1989;  Webster  and  Oliver,  2007].  Furthermore,  it  deals  with  a 
methodology  to  quantify  spatial  uncertainty.  Next,  the  current  applications  and  theoretical  developments 
dealing  with  the  field  of  geostatistics  are  summarized: 

•  The  main  applications  deal  with  soil  sciences:  identifying  chemical  and  physical  soil  properties  (e.g. 
moisture,  salinity,  minerals,  pH,  etc.)  [Basaran  et  ah,  2011;  Paul  and  Cressie,  2011].  Other 
applications  include:  agriculture,  mining,  landscape  ecology  (CO2,  Ozone,  radiation),  and 
manufacturing  problems  [Srivastava,  2013;  Tardic  et  al.,  2014;  Tsui  et  ah,  2013;  Volpi  et  ah,  2014]. 

•  Comparison  of  kriging,  cokriging  methods,  and  other  similar  kriging-based  methods  is  discussed  in 
[Basaran  et  ah,  2011;  Hosseini  et  ah,  2014;  Tsui  et  ah,  2013]. 

•  Reducing  the  computation  cost  of  kriging  for  large  spatial  datasets  is  addressed  in  [Cressie  and 
Johannesson,  2008;  Cressie  and  Kang,  2010]. 

•  Creating  surrogate  models  in  order  to  reduce  the  computation  burden  of  original  physical  models  (i.e. 
dynamic  kriging),  see  for  example  [Volpi  et  al.,  2014;  Zhao  et  ah,  2011;  Zhao  et  ah,  2013]. 

8.4.3.  NRMM 

Pioneering  work  was  developed  by  Lessem  and  others  [Lessem  et  al.,  1992;  Lessem  et  al.,  1993;  Lessem  et 
al.,  1996]  and  constituted  a  significant  contribution  to  convert  NRMM  from  a  deterministic  framework  to  a 
stochastic  one.  In  that  approach,  input  parameters  to  the  NRMM  were  randomly  generated  according  to  a 
given  range,  and  after  Monte  Carlo  simulation  an  output  was  provided  in  terms  of  the  nominal,  maximum  and 
minimum  speeds  for  a  given  scenario.  Uncertainty  was  simulated  by  means  of  a  fixed  range  for  every  input 
parameter  of  the  NRMM.  Those  ranges  were  assigned  by  expert  opinion.  The  ultimate  output  of  Lessem’s 
work  was  a  deterministic  GO/NOGO  map  based  on  the  minimum  value  in  the  expected  range  of  speed.  That 
is,  if  that  minimum  value  is  zero  (representing  vehicle  entrapment)  that  region  was  marked  as  NOGO.  As 
remarked  in  [Priddy,  1995],  Lessem’s  intent  was  to  demonstrate  the  stochastic  forecasting  concepts  rather 
than  to  reflect  accurate  output  variability. 

Another  significant  step  in  the  NRMM  context  was  addressed  in  [Priddy,  1995].  In  that  research,  the  authors 
explained  different  models  to  estimate  certain  parameters  dealing  with  the  vehicle -terrain  interaction  such  as 


UNCLASSIFIED:  Distribution  Statement  A.  Approved  for  public  release;  distribution  is  unlimited 


39 


slip,  motion  resistance,  vehicle  cone  index,  and  drawbar  pull.  This  study  concludes  how  well  the 
(deterministic)  NRMM  performed  with  actual  data  and  proposed  new  prediction  equations  to  account  for 
variability  in  the  cross-country  traction  empirical  relationships. 


8.5  OVERALL  FRAMEWORK  OF  PROPOSED  ARCHITECTURE 

8.5.1  Digital  Terrain  Modeling 

The  process  of  natural  terrain  modeling  starts  with  a  set  of  sparse  measurements  obtained  using  a  remote 
sensor  for  a  terrain  region  of  interest  (see  Figure  8-2).  Typically  these  sensors  are  mounted  on  a  vehicle  or  on 
satellites.  In  any  of  those  scenarios,  both  variable  resolution  (or  small  resolution)  and  irregular  density  of  data 
(occlusions)  are  inevitable.  This  issue  leads  to  non-uniformly  spaced  data.  Therefore,  a  useful  first  step  to 
simulating  the  performance  of  a  vehicle  over  such  terrain  deals  with  generating  a  continuous  surface.  This 
point  is  solved  by  interpolating  the  unknown  height  at  some  uniform  grid  node  or  continuous  surface.  There 
are  many  known  interpolation  methods,  see  [Detweiler  and  Ferris,  2010]  for  a  review  of  four  of  the  most 
popular  ones  (mean,  median,  inverse  distance  to  a  power,  and  ordinary  kriging). 

In  the  proposed  framework,  we  are  interested  in  methods  that  provide  not  only  the  elevation  at  areas  where 
there  is  sparse  or  no  data,  but  also,  and  most  importantly,  an  estimate  of  estimation  error,  that  is,  the 
uncertainty  associated  with  that  new  point.  This  is  what  Gaussian  Process  (GP)  regression  yields.  The  main 
drawback  of  a  GP  is  that  its  performance  is  highly  influenced  by  the  kernel  function  chosen  [Ho  et  ah,  2011]. 
A  particular  version  of  a  GP  in  the  field  of  Geostatistics  is  kriging.  Kriging  produces  an  interpolation  function 
based  on  a  covariance  or  variogram  model  derived  from  the  data  rather  than  an  a  priori  model  of  the 
interpolating  function.  This  fact  mitigates  the  effect  of  choosing  a  general-purpose  kernel  function  as  in  GP. 

Once  a  continuous  surface  is  obtained,  stochastic  simulation  of  the  performance  of  a  vehicle  over  such  terrain 
can  be  performed.  However,  depending  on  the  kind  of  simulations  and/or  the  computational  resources  a  more 
compact  mathematical  model  of  the  terrain  profile  may  be  desired.  This  step  is  mainly  found  in  the 
automobile  industry  where  simulations  deal  with  suspension  loading  conditions,  chassis  fatigue,  etc. 
[Chemistruck  et  ah,  2013;  Ma  et  ah,  2013]. 
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Non*uniformly  spaced  data 
(or  data  with  occlusions] 


INTERPOLATION  I  Gaussian  Process/ Kriging,  Polynomial  interpolation, 
_ Inverse  distance,  Splines,  etc. 


Uniformly  spaced  data  / 
continuous  surface 


OUR  SCOPE:  Stochastic 
mobility  prediction  of 
ground  vehicles 


Figure  8-1.  Schematic  view  of  the  different  steps  dealing  with  digital 

terrain  modeling 


8.5.2.  Stochastic  Mobility  Prediction 

Figure  8-3  shows  the  methodology  of  the  proposed  architecture.  Initially,  a  DEM  is  obtained  related  to  the 
region  of  interest.  After  that,  a  reduced-order  representation  of  the  DEM  points  is  obtained  via  a  subsampling 
approach.  This  reduced-order  representation  is  required  in  order  to  enable  an  affordable  computation  of  the 
variogram  and  kriging  method.  Once  a  set  of  representative  points,  in  terms  of  the  variogram  and  elevation 
profile,  are  selected,  the  ordinary  kriging  method  is  applied.  This  procedure  yields  a  model  of  the  terrain  at  a 
finer  resolution.  This  model  can  be  used  for  statistical  simulation  since  each  interpolated  point  has  an 
uncertainty  associated  with  it  (i.e.  the  kriging  variance).  After  that,  two  possible  results  can  be  obtained:  a 
mobility  map  or  a  route  planning  result.  Those  two  results  are  explained  in  Section  5. 

The  main  features  of  this  architecture  are  summarized  as  follows: 

•  Global  path  planning  is  considered  rather  than  local  path  planning  (i.e.  planning  in  the  close 
vicinity  of  the  vehicle).  From  the  decision-maker’s  point  of  view,  this  feature  is  important 
because  it  provides  an  ability  to  make  movement  decisions  over  large  spatial  regions. 

•  The  main  source  of  uncertainty  comes  from  surface  geometry  (elevation)  and  soil  properties. 
The  first  one  is  framed  within  the  context  of  global  path  planning;  the  second  one  will  deal 
with  stochastic  GO/NOGO  maps. 

•  This  solution  does  not  result  in  a  binary  answer,  i.e.  the  path  is  traversable  or  not;  instead 
statistical  data  supporting  each  decision  is  given. 

•  An  efficient  solution  can  be  obtained  and  has  been  demonstrated  on  a  standard-performance 
laptop. 
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Figure  8-2.  Schematic  view  of  the  steps  carried  out  in  the  proposed  architecture  for 
predicting  the  mobility  of  a  ground  vehicle  over  a  large  region  (>  5  x  5  [km^]) 


8.6  POTENTIAL  SOLUTIONS  TO  ELEMENTS  OE  PROPOSED 
ARCHITECTURE 

This  section  introduces  some  potential  solutions  to  the  different  elements  of  the  proposed  stochastic  mohility 
prediction  architecture.  In  particular,  a  methodology  based  on  global  path  planning  is  formulated  in  order  to 
cope  with  route  planning  in  the  presence  of  elevation  uncertainty.  Additionally,  a  novel  segmentation -based 
algorithm  is  proposed  to  deal  with  the  common  issue  of  non-stationary  variogram  models.  After  that,  a 
mobility  analysis  solution  is  described  in  terms  of  uncertainty  in  soil  properties.  A  novel  approach  is 
introduced  to  cope  with  uncertainty  in  Bekker-Wong  parameters.  Then,  the  Bekker-Wong  model  is  applied 
leading  to  a  stochastic  mobility  map  where  decisions  are  made  in  terms  of  the  maximum  drawbar  pull  force 
that  a  vehicle  can  generate. 

8.6.1.  Route  Planning 

This  element  of  the  suggested  methodology  deals  with  analyzing  the  performance  of  a  vehicle  moving 
between  two  given  points,  a  starting  point  and  goal  point,  considering  a  model  of  the  terrain  and  its  associated 
uncertainty.  The  D*  algorithm  has  been  employed  in  order  to  obtain  an  optimal  route  between  the  starting  and 
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goal  points  [Stentz,  1995].  In  this  research,  three  metrics  have  been  considered  for  obtaining  such  a  route.  The 
first  metric  finds  the  shortest  route  between  the  starting  point  and  goal  point.  For  that  purpose,  the  D*  cost 
function  considers  the  Euclidean  distance  between  points  (in  an  x-y  plane).  Notice  that  in  this  case, 
uncertainty  is  not  considered.  The  second  route  is  obtained  as  the  shortest  distance  between  the  starting  and 
goal  points  but  also  minimizing  the  uncertainty.  That  is,  the  variance  associated  with  each  point  is  also 
considered  in  the  D*  cost  function.  Finally,  the  last  route  represents  the  shortest  route  between  the  starting  and 
goal  points  but  also  minimizing  the  slope  between  points.  Finally,  the  optimal  route  is  given  in  terms  of  some 
performance  indices  (e.g.  the  shortest  path,  the  path  with  the  lowest  uncertainty,  the  flattest  route,  etc.). 

8.6.2.  Segmentation-Based  Local  Variogram  Models  for  Ordinary  Kriging 

Notice  that  ordinary  kriging  is  based  on  the  assumption  of  a  stationary  variogram.  This  requirement  means  the 
mean  and  variance  of  such  variogram  is  finite  and  constant  in  the  area  under  investigation  [Fisher,  1998].  In 
practice,  this  assumption  is  not  always  ensured  [Atkinson  and  Lloyd,  2007;  Chen  and  Li,  2012;  Lloyd  and 
Atkinson,  1999].  This  fact  is  especially  noticeable  when  a  global  variogram  intends  to  capture  the  nature  of  a 
heterogeneous  region. 

In  other  to  solve  the  issue  of  non-stationarity,  different  approaches  have  been  proposed  in  the  literature.  Such 
approaches  can  be  grouped  in  the  following  three  categories  [Zhang  et  ah,  2014]:  (i)  locally  adaptive  kriging 
involves  predicting  and  modeling  the  local  experimental  variogram  and  using  the  coefficients  of  the  locally 
fitted  model  in  (the  local)  kriging;  (ii)  surface  deformation  aims  to  distort  a  surface  such  that  a  stationary 
variogram  results  from  data  in  transformed  space;  (iii)  segmentation  involves  dividing  the  region  of  interest 
into  smaller  segments  within  the  variogram  that  can  be  considered  stationary,  thus  allowing  for  local 
application  of  geostatistical  optimal  sampling  design  in  their  study. 

Segmentation  constitutes  the  most  commonly  used  approach;  see  for  example  [Atkinson  and  Lloyd,  2007; 
Chen  and  Li,  2012;  Lloyd  and  Atkinson,  1999].  Some  of  those  references  divide  the  region  of  interest  by 
using  a  predefined  template  or  rule,  for  example,  dividing  the  environment  into  4-subregions  each  time  non- 
stationarity  is  found  during  the  segmentation  process  [Chen  and  Li,  2012].  The  main  drawback  of  this 
approach  is  that  it  does  not  take  advantage  of  the  properties  of  the  local  variograms  in  order  to  increase  the 
accuracy  of  the  segmentation  step.  In  contrast,  in  the  works  [Atkinson  and  Lloyd,  2007 ;  Lloyd  and  Atkinson, 
1999],  a  clustering  segmentation  algorithm  is  employed.  The  metrics  on  which  the  segmentation  is  based  is 
the  fractal  dimension  [Klinkenberg  and  Goodchild,  1992].  The  main  limitation  of  this  method  is  that  the 
fractal  dimension  cannot  be  applied  when  the  region  of  interest  does  not  fulfill  Brownian  properties  [Kroese 
and  Botev,  2014]. 

The  proposed  approach  makes  use  of  both  the  fractal  dimension  and  elevation  range  as  metrics  in  the 
segmentation  step.  This  method  can  be  applied  to  any  type  of  man-made  or  natural  terrain  profile.  Notice  that 
elevation  range  constitutes  a  well-known  metric  in  the  field  of  Geomorphology;  it  has  been  mainly  used  to 
identify  and  classify  terrains  [Evans,  2012;  Saadat  et  ah,  2008]. 

8.6.3.  Mobility  Map  Based  On  Soil  Uncertainty 

A  stochastic  mobility  map  is  generated  via  Monte  Carlo  simulation.  In  particular,  for  a  given  soil  region  n 
realizations  are  obtained  for  each  BW  parameter  according  to  its  associated  Gaussian  (or  other)  distribution. 
Thus,  this  process  leads  to  n  values  for  the  drawbar  pull  force  (DP),  obtained  using  a  Bekker-Wong  vehicle- 
terrain  interaction  model.  If  in  a  given  cell  the  DP  is  higher  than  the  vehicle  can  actually  reach,  such  cell  is 
marked  as  no  traversable  (NOGO).  A  cell  is  considered  traversable  when  the  DP  of  m  runs  is  greater  than  a 


UNCLASSIFIED:  Distribution  Statement  A.  Approved  for  public  release;  distribution  is  unlimited 


43 


given  threshold  (m  >  8,  where  8  is  a  given  confidence  interval). 


We  note  that  there  does  not  exist  an  exhaustive  global  database  for  all  parameters  of  the  Bekker-Wong  model, 
for  all  the  soil  types.  To  be  able  to  assign  a  significant  value  to  unknown  soil  parameters,  for  each  soil  type  a 
procedure  based  on  interpolation  from  documented  values  of  similar  soil  parameters  has  been  implemented.  In 
particular,  the  value  of  the  parameter  x  for  the  soil  type  i  has  been  obtained  by  solving  the  following  equation 
for  M  random  values  for  each  neighboring  point 


Xi  =mean  {X. ) ,  5  (x . )  =std  {X. ) , 


(1) 


where  w  is  given  as  the  inverse  of  the  distance  between  the  centroids  of  the  cells  in  the  USDA  triangle 
[USDA,  1987].  The  value  R  comes  from  generating  M  random  values  within  the  normal  distribution 
associated  to  each  soil  type  for  this  parameter.  The  procedure  followed  to  obtain  this  normal  distribution  is 
explained  subsequently. 

Notice  that  the  goal  of  this  potential  solution  is  to  represent  the  variability  in  the  Bekker-Wong  parameters  by 
means  of  a  normal  distribution.  A  second  point  deals  with  removing  the  presence  of  outliers  in  those  physical 
experiments.  The  decision  adopted  regarding  the  outlier  removal  is  mainly  based  on  our  experience  and  it  has 
been  found  after  testing  and  comparing  different  metrics.  Eventually,  an  outlier  is  detected  when  it  is  out  of 
the  following  range 


( 


medianiV) 

r 


r*  medianiV)), 
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where  median(V)  represents  the  median  value  for  all  the  values  related  to  a  particular  parameter  and  a  soil 
type,  p  is  an  experimental  parameter  manually  tuned  in  order  to  increase  or  decrease  the  range. 


8.7.  PROOF  OF  CONCEPT  RESULTS 

This  section  introduces  some  illustrative  examples  demonstrating  the  suitability  of  the  proposed  architecture. 
All  these  experiments  are  based  on  digital  elevation  models  of  real  scenarios.  In  particular,  the  7.5-Min  USGS 
format  has  been  considered,  that  is,  the  spatial  resolution  of  the  models  is  30  meters.  The  code  has  been 
implemented  in  Matlab  using  the  Geostatistical  toolbox  mGstat  (http://mgstat.sourceforge.net). 

8.7.1.  Route  Planning 

We  have  demonstrated  the  suitability  of  the  proposed  stochastic  mobility  prediction  approach  over 
relatively  large  regions  [>  5  x  5  [km2]).  Figure  8-3  shows  the  performance  of  the  route  planning 
approach  over  two  different  scenarios.  Figure  8-3a  displays  a  deterministic  terrain  profile  illustrating 
the  minimum  slope  between  points  [8-neighbors  to  each  point).  In  this  sense,  a  path  going  through  a 
brighter  region  [yellow)  would  mean  a  flatter  route  [small  variation  in  the  elevation  between  one  point 
and  its  neighbor).  On  the  other  hand,  hazards  such  as  high  slopes  are  represented  by  blue  or  red  color. 
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that  is,  the  difference  in  elevation  between  one  point  and  its  neighbors  is  larger  than  in  a  brighter 
region.  Notice  that  positive  values  [red  color)  mean  positive  slopes  [the  vehicle  would  pitch  up),  and 
negative  values  [blue  color)  represent  negative  slopes  [the  vehicle  would  pitch  down). 

Figure  8-3b  shows  the  min-distance  and  the  min-uncertainty  routes  for  the  Sahara  desert  region.  As 
expected,  the  shortest  route  [straight-line)  corresponds  to  the  min-distance  line  [red  line).  The  min- 
uncertainty  route  considers  the  variance  of  the  elevation  [uncertainty  obtained  from  kriging).  For  that 
reason  this  route  passes  as  close  as  possible  to  the  original  sampled  points  [black  dots). 


(a)  Min-slop  route.  Airport  Lake  (Death  Valley, 
CA,  USA) 


(b)  Min-distance  and  Min-uncertainty  routes. 
Sahara  desert  ( Chad,  Africa) 


Figure  8-3.  Routes  obtained  using  the  global  path  planner.  The  mesh  represents  the  terrain 
model  considering  nominal  elevations  (kriging  estimations). 

8.7.2.  Segmentation-Based  Local  Variogram  Models  For  Ordinary  kriging 

The  main  goals  regarding  this  work  are  to  increase  kriging  accuracy  and  reduce  computation  time.  The 
suitability  of  the  proposed  method  has  been  demonstrated  with  heterogeneous  scenarios,  i.e.  scenarios  that 
include  natural  Brownian-like  terrain  profiles,  natural  non-Brownian-like  terrain  profiles,  and  scenarios 
combining  natural  and  man-made  regions.  In  all  those  cases,  the  standard  deviation  of  the  kriging  variance  is 
smaller  when  the  local  variograms  are  considered  instead  of  the  global  variogram,  resulting  in  smaller 
uncertainty  in  the  new  interpolated  points.  Furthermore,  computation  time  has  been  reduced  in  the  proposed 
approach.  For  instance,  for  a  given  region,  the  computation  time  following  the  traditional  approach  (i.e. 
computing  a  global  variogram  over  the  entire  environment  of  interest)  is  approximately  1  hour  on  COTS 
laptop;  considering  local  regions  and  local  variograms,  the  computation  time  for  the  same  environment  is  less 
than  2  minutes. 

Figure  8-4a  shows  the  result  obtained  after  applying  the  segmentation-based  approach  to  an  environment 
composed  of  natural  and  man-made  regions,  Hyannis  Village  (Barnstable,  MA,  USA).  Figure  8-4b  shows  the 
variograms  of  the  original  30  local  regions. 
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Subregions  with  region  growing,  No.  of  classes:  8 


Variogram  for  30  clusters 


(a)  Regions  after  applying  the  segmentation 
based  on  the  fractal  dimension  and  elevation 
range 

Figure  8-4.  Hyannis  Village  (Barnstable,  MA,  USA). 


(b)  Local  variograms  for  all  the  regions  before 
merging  those  regions  with  similar  features 


8.7.3.  Mobility  Map  Based  On  Soil  Uncertainty 

As  previously  explained,  a  novel  methodology  has  been  proposed  in  order  to  represent  each  parameter  in  the 
Bekker-Wong  model  for  each  soil  type  in  the  USDA  soil  system  according  to  a  Gaussian  distrihution.  Figure 
8-5a  shows  an  example  of  such  Gaussian  distrihution,  in  this  case,  the  internal  friction  angle  for  the  12  soil 
types  in  the  USDA  classification  system.  Soil  parameter  data  was  collected  from  a  variety  of  published 
sources  in  the  open  literature.  It  bears  mentioning  that  in  order  to  avoid  a  misrepresentation  of  the  Gaussian 
distribution  a  filter  was  designed  in  order  to  remove  outliers  from  the  calculation.  An  example  of  such  filter  is 
shown  in  Figure  8-5b.  In  particular,  all  the  measurements  associated  to  the  cohesion  of  sandy  loam  are 
plotted,  but  only  those  regions  within  a  certain  range  (solid  circles)  are  used  for  determining  the  Gaussian 
distribution. 
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USDA  class 


C(>0«12). Orl0nil:a.FiHBf:  Id 


(a)  Internal  friction  angle  (b)  Cohesion,  sandy  loam 

Figure  8-5.  Variability  in  Bekker-Wong  parameters  and  filter  designed  to  remove  outliers 


On  the  other  hand,  Figure  8-6  shows  a  Matlah  GUI  implemented  in  order  to  perform  interactive  simulations 
regarding  soil  trafficahility.  In  this  case,  a  random  surface  is  generated  and  three  soil  types  are  assigned  to 
three  different  regions.  Then,  a  mobility  map  is  obtained  according  to  the  maximum  drawbar  pull  force 
introduced  by  the  user. 


NextCen  NRMM 


Variables  Credits 


Surface  reconstruction  Terrain  profile  ’’  Soil  trafficability  ]  Path  planning  Uncertainty 


Figure  8-6.  Matlab  GUI  implemented  in  order  to  perform  stochastic  mobility  analysis 
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8.8.  RECOMMENDATIONS  AND  OPEN  QUESTIONS 


Based  on  this  study,  the  following  recommendations  are  made: 

•  Any  extension  of  NRMM  in  terms  of  stochastic  mobility  prediction  should  allow  for  consideration  of 
uncertainty  in  elevation  as  well  as  in  soil  physical  properties.  Notice  that  uncertainty  in  elevation  is  in 
fact  present  in  any  DEM  (uniform  continuous  model  derived  from  sparse  data),  and  uncertainty  in 
terrain  properties  is  also  expected  due  to  the  physical  variability  of  natural  terrain. 

•  As  evidenced  in  this  work,  computation  time  constitutes  a  key  factor  that  must  be  considered  in  the 
development  of  the  new  NRMM.  In  this  sense,  any  new  proposal  should  focus  on  efficient 
algorithms.  Notice  that  avoiding  this  recommendation  may  lead  to  practically  infeasible  solutions. 

•  It  is  desirable  from  a  stochastics  perspective  to  base  vehicle -terrain  interaction  on  the  Bekker-Wong 
model,  as  these  models  are  compatible  with  numerous  multi-body  dynamic  simulation  codes.  Other 
well-known  solutions  such  as  Cone  Index  (Cl)  do  not  have  this  property. 

After  an  analysis  of  the  state  of  the  art  and  the  work  performed  in  the  framework  of  this  effort,  the  following 
concerns  are  still  open  to  debate: 

•  Soil  moisture  constitutes  an  essential  climate  variable  that  deals  with  the  level  of  water  diffused  as 
vapor  or  condensed  in  soil.  Even  though  it  seems  that  soil  moisture  “implicitly”  appears  in  Bekker- 
Wong  parameters,  there  is  a  lack  of  experimental  data  relating  soil  moisture  to  those  parameters  for 
all  the  soil  types  appearing  in  the  USDA  soil  classification  system.  A  possible  solution  to  this 
problem  would  require  performing  experiments  with  the  bevameter  technique  under  different  soil 
moisture  levels  and  finding  some  kind  of  relationship  between  Bekker-Wong  parameters  and  the  level 
of  water  on  such  soils. 

•  There  is  not  a  clear  answer  to  what  is  the  most  appropriate  spatial  resolution  in  order  to  perform  a 
reliable  stochastic  mobility  prediction  analysis.  It  is  not  known  whether  any  detailed  study  on  this 
issue  has  been  performed.  It  appears  that  spatial  resolution  of  data  for  3D  terrain  models  should  be 
dependent  on  the  size  of  the  vehicle,  the  variability  of  the  terrain,  and  on  the  nature  of  any  natural  or 
man-made  obstacles  that  the  vehicle  must  negotiate. 
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Chapter  9  -  THEME  4:  INTELLIGENT  VEHICLES 

Abhinandan  Jain 


9.1  GOALS  AND  DELIVERABLES 

The  Goals  of  Theme  4  are  to  define  a  Next-Gen  NRMM  approach  and  requirements  for  mobility  assessment  for 
intelligent  vehicles.  For  the  purposes  of  this  discussion,  an  intelligent  vehicle  is  assumed  to  he  one  without  a 
human  driver  onboard,  and  operated  with  a  combination  of  on-board  intelligence,  remote  operators  and  shared- 
control  resources.  The  vehicle  itself  may  have  onboard  passengers,  and  may  be  operated  singly  or  as  part  of  a 
group  of  vehicles.  Within  this  section,  we  adopt  the  following  acronyms  to  distinguish  between  the  Next-Gen 
NRMM  for  manned  and  intelligent  (unmanned)  vehicles: 

•  NRMM(H)  -  Next-Gen  NRMM  for  manned  vehicles,  i.e.  vehicles  with  onboard  human  driver 

•  NRMM(I)  -  Next-Gen  NRMM  for  intelligent  vehicles  (w/o  onboard  human  driver) 

Historically,  the  focus  of  NRMM  has  been  on  manned  vehicles  alone,  and  hence  has  been  synonymous  with 
NRMM(H).  However,  with  the  rapid  emergence  of  intelligent  vehicle  capabilities,  the  need  for  NRMM(I)  has 
become  evident,  and  we  seek  here  to  define  ideas  and  approaches  that  are  pertinent  to  its  development.  While  it 
is  expected  that  NRMM(I)  will  leverage  and  benefit  from  NRMM(H)  development,  we  focus  here  specifically 
on  NRMM(I)  since  the  development  of  NRMM(H)  is  covered  in  considerable  detail  in  the  rest  of  this  document. 

Some  of  the  questions  and  topics  addressed  are  as  follows: 

•  Define  intelligent  vehicle  classes  and  mobility  types 

•  Define  range  of  operational  environments 

•  What  characteristics  of  intelligent  vehicles  are  pertinent  to  NRMM? 

•  What  is  common  and  different  from  manned  vehicle  NRMM? 

•  What  NRMM  output  products  are  appropriate  for  intelligent  vehicles? 

•  What  approaches  can  we  use  to  make  performance  metrics  quantitative? 

•  Identify  methods  specific  to  intelligent  vehicles 

•  Identify  tool  needs  for  intelligent  vehicles 

•  Identify  current  capabilities  and  gaps 

The  members  of  Theme  4  are  the  following: 
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Jain,  Abhinandan,  Leader 

USA 
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Letherwood,  Michael 
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Ward,  Derek 
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9.2  WHAT  IS  DIFFERENT  ABOUT  INTELLIGENT  VEHICLES? 


Before  plunging  into  NRMM(I)  capability  development,  and  how  it  relates  to  the  NRMM(H)  capability,  we 
begin  by  reviewing  the  characteristics  of  intelligent  vehicles  that  distinguish  them  from  manned  vehicles.  The 
areas  of  differentiation  include  (a)  the  types  of  vehicle  mobility;  (b)  variations  in  their  environment  of 
operation;  and  (c)  their  control. 

9.2.1  Variety  of  Mobility  Types 

Traditionally  NRMM  has  focused  on  large  wheeled  and  tracked  vehicles  with  manned  drivers.  The  family  of 
intelligent  vehicles  include  unmanned  versions  of  these  vehicles  as  well  as  others  such  as  (see  Figure  9-1): 

•  Large  wheeled/tracked  vehicles:  These  are  unmanned  versions  of  the  traditional  large 
wheeled/tracked  vehicles.  These  may  be  operated  individually  or  be  part  of  a  mixed  convoy  of 
manned  and  unmanned  vehicles. 

•  Small  robots:  A  number  of  portable,  small  wheeled/tracked  vehicles,  e.g..  Talons,  Pacbots,  are 
already  in  active  use  in  operational  settings  and  are  emerging  as  an  important  new  class  of  vehicles. 

•  Legged  robots:  While  wheeled  and  tracked  vehicles  are  the  dominant  class  of  mobile  vehicles,  they 
can  operate  only  over  smooth  or  moderately  rough  terrains.  Legged  vehicles  (eg.  Big  Dog)  are  being 
developed  for  rough  terrain  environments. 

•  Bipedal  Humanoids:  Humanoid  robots  (eg.  Petman,  Atlas)  are  another  area  of  development  where 
the  limbs  can  be  used  as  support  legs  as  well  as  for  manipulation  tasks. 

•  Emerging  technologies:  There  are  ongoing  technological  developments  involving  non-traditional 
platforms  such  as  climbing/insect  robots,  as  well  as  ones  involving  coordinated  mobility  and 
manipulation.  Moreover  vehicles  can  be  operated  as  part  of  multi-vehicle  convoys,  cooperating 
vehicles  and  robots,  loosely  coupled  swarms  etc.  Multi-modal  mobility  such  as  for 
amphibious/ground  operation  or  involving  limbed/wheel  platforms  are  also  relevant  for  NRMM(I). 


Figure  9-1:  Example  of  a  variety  of  ground  vehicle  platforms. 


9.2.2  Operational  Environments 

Intelligent  vehicles  can  operate  in  the  following  environments  (see  Figure  9-2): 
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•  On-road,  urban:  Operation  over  roads,  while  following  traffic  rules  (e.g.,  lane -following,  lane- 
change,  traffic  signals,  speed  limits,  over  passes,  tunnels  etc.).  Maneuvering  in  the  presence  of  other 
traffic  as  well  as  pedestrians. 

•  Off-road:  Operation  in  off-road  areas  under  a  variety  of  terrain  types  and  vegetation;  unstructured, 
uncertain  conditions  with  hazards  and  impassable  areas. 

•  Building  interiors:  Operation  within  building  interiors  or  other  structures,  and  navigating  doors, 
stairs,  hallways,  railings  etc. 


Figure  9-2:  Ground  vehicles  operating  in  off-road,  urban  and  indoor  environments. 


9.2.3  Control  Options 

In  general,  the  operation  of  a  vehicle  can  be  viewed  as  involving 

1 .  Onboard  human  driver 

2.  Onboard  autopilot/intelligence 

3.  Remote  human  driver 

4.  Remote  autopilot/intelligence 

While  option  (1)  is  the  focus  of  NRMM(H),  options  2-4  characterize  intelligent  vehicle  control  and  operation 
as  described  in  the  examples  below  (see  Figure  9-3): 

•  Intelligent  vehicles  have  no  onboard  human  driver,  but  can  be  operating  with  other  human  driven 
vehicles  or  in  convoys  with  other  UGVs. 

•  They  typically  have  remote  operators  and  resources.  Control  modes  can  include  low-level 
teleoperation,  to  shared  control,  to  full  autonomy.  Closed  loop  control  can  be  impacted  by  bandwidth 
and  latency  limitations  over  the  communication  link. 

•  A  key  characteristic  of  intelligent  vehicles  is  the  presence  of  an  onboard  sensor  suite  and  use  of 
onboard  software  and  algorithms  for 

-  Sensor  fusion,  localization,  state  estimation,  handling  of  noise/drop  outs,  obstacle  detection, 
situational  awareness,  map  building 

-  Locomotion,  obstacle  avoidance,  slippage  detection,  model  predict  motion  control  algorithms 

-  Legged  -  self  balancing,  foot  placement,  walking  gaits,  manipulation  etc. 

-  Executive  for  real-time  coordination  and  control,  shared  control  interface 

-  Planning/executive  layer  for  deliberative  long  term  motion  and  path  planning,  vehicle  fault 
diagnosis/recovery 
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Figure  9-3:  Vehicle  intelligence  involves  multiple  on-board  sensors,  autonomy  algorithms,  and  interaction  with 

remote  operators  and  resources. 


The  performance  of  an  intelligent  vehicle  can  he  inferior  (due  to  less  sophisticated  sensing,  decision/planning 
and  control)  as  well  as  superior  (due  to  no  fatigue  or  distractions,  faster  processing  of  information,  more 
sensors)  in  comparison  with  the  performance  of  a  manned  vehicle.  For  a  hroad  overview  of  the  growing 
presence  and  importance  of  onboard  autonomy  -  and  the  challenges  they  represent  across  DoD  applications 
please  see  references  [Defense  Science  Board,  2012;  Air  Force  Research  Laboratory,  2013;  Office  of 
Technology  Intelligence,  2015]. 

9.2.4  Vehicle  Intelligence  Challenges 

The  characterization  of  the  performance  of  intelligent  vehicles  required  for  NRMM(I)  presents  several 
additional  challenges  over  performance  characterization  for  manned  vehicles.  Some  of  these  are: 

•  Vehicle  intelligence  is  an  amorphous  concept:  There  is  no  standard  or  settled  definition  for  onboard 
intelligence.  There  is  significant  variance  in  intelligence  architectures  as  well  as  capabilities  -  even 
for  the  same  vehicle  hardware  platform.  Performance  assessment  methods  have  to  handle  such 
variability 

•  Lack  of  performance  metrics  for  autonomous  systems:  While  quantitative  performance  metrics  are 
essential  for  NRMM,  such  metrics  are  seriously  lacking  in  the  vehicle  intelligence  area.  On  the  one 
hand  the  difficulty  is  in  defining  metrics  that  span  performance  over  the  large  space  of  operational 
conditions  and  environments,  and  on  the  other  is  the  paucity  of  analytical  techniques  for  the 
characterization  of  the  performance  of  rule  based  modules.  As  a  result,  metrics  often  are  based  on 
empirical  measurements  over  a  small  sample  set. 

•  Vehicle  intelligence  is  a  rapidly  evolving  area:  Vehicle  intelligence  technology  is  rapidly  evolving 
-  on  both  the  hardware  and  software  fronts.  It  is  essential  that  the  techniques  developed  for  NRMM(I) 
be  able  to  scale  and  handle  performance  assessment  from  such  new  and  emerging  intelligent  vehicle 
capabilities,  or  else  risk  rapid  obsolescence.  While  “one  of’  solutions  for  NRMM(I)  may  be  expedient 
they  may  not  be  useful  over  time. 

•  Vehicle  intelligence  is  not  all  or  nothing:  Often  vehicle  “intelligence”  and  “autonomy”  are  viewed 
as  on  or  off  capabilities.  This  is  rarely  the  case  in  reality.  The  more  typical  situation  is  that  of  sliding 
autonomy.  That  is,  onboard  intelligence  modules  typically  provide  a  broad  range  of  modes  and 
options  to  select  between  different  levels  of  autonomy,  where  selective  features  can  be  disabled  or 
degraded  as  needed.  An  important  consequence  of  this  is  that  the  primary  goal  of  an  NRMM(I) 
capability  is  not  so  much  to  provide  GO/NOGO  guidance  for  vehicle  intelligence,  and  instead  is  to 
provide  guidance  on  the  level  of  autonomy  to  use  for  the  best  performance  and  risk  outcome  for  the 
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mission  at  hand. 

•  Performance  evaluation  is  significantly  more  complex:  One  of  the  challenges  with  developing 
performance  measurement  techniques  for  vehicle  intelligence  is  the  high  dimensional  state  space 
associated  with  the  intelligence  algorithms  together  with  the  large  dimensionality  of  representations 
for  unstructured/uncertain  environments.  Such  large  dimensional  comhinatorics  is  difficult  and 
impractical  to  handle  using  standard  techniques. 

•  Coupling  between  vehicle  dynamics  and  intelligence  is  poorly  understood:  By  and  large,  the 
intelligence  development  focuses  on  the  sensing,  kinematic  and  geometric  characteristics  of  the 
vehicle.  While  this  may  he  appropriate  for  quasi-static  or  slowly  moving  vehicles,  such  approaches 
are  inadequate  for  vehicles  moving  at  even  moderate  speed  where  the  vehicle  dynamics  plays  an 
important  role  in  its  performance.  Significant  interaction  between  the  vehicle  dynamics  and  vehicle 
intelligence  communities  is  essential  for  the  development  of  autonomy  capabilities  for  dynamic 
vehicles  as  well  as  the  performance  assessment  capabilities  needed  for  NRMM(I). 

•  Leveraging  classical  NRMM(H)  for  human  driven  vehicles  is  desirable:  There  is  parallel 
development  of  the  next  generation  NRMM(H)  capability  for  manned  vehicles  that  ideally  NRMM(I) 
should  be  able  to  leverage.  This  requires  a  good  understanding  of  the  coupling  between  the 
NRMM(H)  and  NRMM(I)  capabilities  to  avoid  duplication,  as  well  as  to  influence  the  development 
of  NRMM(H)  so  that  it  includes  interfaces  and  supports  performance  data  products  required  by 
NRMM(I). 

•  Off-line  as  well  as  in-the-field  NRMM  usage  needs:  Use  during  operations  requires  the  timely 
generation  of  performance  assessment  results.  This  imposes  additional  speed  requirements  on 
NRMM(I)  usage. 

9.2.5  Does  Vehicle  Dynamics  Impact  Intelligence  Performance? 

As  mentioned  earlier,  the  coupling  between  vehicle  dynamics  and  vehicle  intelligence  performance  is 

poorly  understood  and  often  not  seriously  considered  during  intelligence  design,  development  and  evaluation. 

In  fact  there  is  a  strong  connection  between  them.  Some  examples  of  the  coupling  between  vehicle  dynamics 

and  the  performance  of  the  onboard  intelligence  are: 

•  While  the  effect  of  ride  roughness  and  vibration  on  drivers  is  not  relevant  for  UGVs  (unless  there  are 
onboard  passengers),  ride  roughness  and  vibration  can  degrade  sensor  performance.  The  impact  can  lead 
to  dropouts  and  increase  in  sensing  error.  Degraded  sensor  performance  directly  impacts  key  intelligence 
functions  such  as  obstacle  detection  and  detection  of  traffic,  pedestrians  and  road  signals  that  onboard 
intelligence  depends  on. 

•  Vehicle  speed  can  also  effects  the  performance  and  update  rates  of  onboard  sensors  used  by  the  onboard 
intelligence.  Moreover,  higher  vehicle  speed  can  reduce  the  time  windows  available  for  the  onboard 
algorithms  (such  as  obstacle  detection  and  avoidance)  to  complete  their  computations  which  adversely 
impact  their  robustness  and  performance. 

•  The  dynamic  behavior  of  vehicles  is  affected  by  vehicle/terrain  interaction  that  results  in  vehicle  slippage. 
Vehicle  slippage  can  introduce  errors  in  the  autonomy  software’s  estimate  of  the  vehicles  state.  Accurate 
knowledge  of  the  vehicle  state  is  critical  information  used  by  the  other  autonomy  algorithms  such  as  for 
situational  awareness  and  motion  planning,  and  slippage  derived  errors  can  significantly  degrade  the 
performance  of  the  autonomy  algorithms.  In  addition,  when  slippage  is  high  (eg.  on  slopes)  proper 
traction  control  needs  to  be  taken  into  account  for  reducing  slippage  for  the  accurate  control  of  the 
vehicle’s  motion. 

•  The  suspension  and  dynamic  properties  of  vehicles  define  their  stability  and  rollover  limits.  These  limits 
need  to  be  taken  into  account  by  onboard  motion  planning  algorithms  for  the  vehicle  during  nominal 
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driving,  lane  change  maneuvers  and  obstacle  avoidance  especially  when  driving  at  modest  to  high  speeds 
in  order  to  ensure  and  safe  and  stable  performance. 

•  Latencies  in  control  action  can  have  a  significant  impact  on  vehicle  dynamics.  This  can  be  an  important 
consideration  since  the  vehicle  control  loop  for  intelligent  vehicles  can  involve  sensor  hardware,  sensor 
data  processing,  state  estimation,  motion  planning  algorithms  as  well  as  communication  and  data 
exchange  with  remote  operators  which  can  all  contribute  to  significant,  and  variable  latency  in  the  control 
action. 


9.3  QUANTITATIVE  FRAMEWORK  FOR  ASSESSING  VEHICLE 
INTELLIGENCE 

The  inputs  for  the  traditional  NRMM(H)  consist  of  models  and  data  for  the  vehicle  platform  and  the  terrain 
environment  the  vehicle  is  to  operate  in.  NRMM(H)  processes  these  together  with  the  mission  scenario 
requirements  and  constraints  to  generate  GO/NOGO  maps  for  the  vehicle,  and  estimates  of  the  attainable 
vehicle  speeds  to  help  guide  the  vehicle  operation.  In  this  context,  the  change  for  NRMM(I)  is  in  the  form  of 
additional  inputs  to  the  process  consisting  of  models  for  the  on-board  and  off-board  shared  intelligence 
resources  (see  Figure  9-4). 


Intelligence  Model  ■  Vehicle  Model  H  Terrain  Model 


V  / 


Figure  9-4:  NRRM(I)  introduces  models  for  vehicle  intelligence  that  need  to  he  included  in  the  prediction  of 

vehicle  performance. 

A  key  requirement  for  the  NRMM(I)  outputs  are  quantitative  metrics  that  provide  actionable  guidance  for  the 
safe  and  optimal  operation  of  the  vehicle  to  meet  the  mission  scenario  objectives.  Some  of  the  existing  efforts 
to  develop  quantitative  assessments  of  semi -autonomous  ground  vehicle  performance  are  described  in 
references  [Haueisen,  2004;  Baylot,  2005;  Richmond,  2009].  Given  the  uncertainty  in  the  model  inputs  to  the 
NRMM  process,  it  is  necessary  that  the  performance  predictions  generated  by  NRMM(I)  be  accompanied 
with  risk  assessment  that  reflect  the  confidence  in  meeting  the  projected  performance.  Operationally,  the  plan 
for  vehicle  mobility  will  have  to  take  into  account  not  only  that  the  vehicle  can  meet  the  desired  objectives, 
but  also  that  the  risks  are  below  the  threshold  acceptable  for  the  mission. 

9.3.1  Intelligence  Levels 

Virtually  all  intelligent  systems  are  designed  to  support  multiple  levels  of  intelligence  that  can  be  selectively 
enabled  during  operations.  This  is  also  referred  to  as  sliding  autonomy.  For  instance  an  intelligent  vehicle 
may  support  manual  operation,  or  operation  with  just  the  onboard  obstacle  detection  turned  on,  or  with  both 
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obstacle  detection  and  obstacle  avoidance  enabled,  or  at  an  even  higher  level  with  autonomous  path  planning 
and  navigation  to  goal  enabled.  These  options  describe  operational  modes  with  increasing  levels  of  onboard 
autonomy.  The  purpose  of  these  multiple  autonomy  level  options  is  to  allow  the  use  of  the  intelligence  mode 
that  best  meets  the  dual  objectives  of  exceeding  performance  needs  while  keeping  risk  below  acceptable 
thresholds  for  a  given  task  and  environment.  As  an  example,  it  is  possible  that  in  environments  with  dense 
clutter,  the  vehicle  may  be  operated  with  only  hazard  detection  mode  on,  with  autonomous  obstacle  avoidance 
being  enabled  only  when  operating  in  less  cluttered  situations.  Similarly  the  remote  human  operator  may 
choose  to  manually  joystick  control  a  vehicle  in  tight  situations,  or  manually  supervise  lane  change  maneuvers 
on  busy  roads.  Even  human  drivers  only  use  cruise  control  on  highways  and  not  on  city  streets  where  the  need 
for  reactive  control  is  higher.  Given  this  context,  the  need  for  NRMM(I)  is  to  generate  data  products  that  can 
assist  the  remote  operator  in  choosing  the  vehicle  intelligence  level  to  best  meet  mission  objectives  from 
scenario  to  scenario. 

Figure  5  below  depicts  an  example  scenario  illustrating  the  use  of  NRMM(I).  In  this  example,  the  mission 
performance  objectives  consist  of  traverse  time,  accuracy  with  which  the  path  is  followed,  and  requirements 
on  the  stability  of  the  vehicle.  For  each  of  these  performance  objectives,  there  are  assumed  to  be  minimum 
performance  requirements,  as  well  as  maximum  acceptable  risk  levels.  The  vehicle  intelligence  is  assumed  to 
support  three  modes,  namely  (A)  pure  manual  control  by  the  remote  operator  with  no  feedback;  (B)  manual 
control  by  the  operator  with  feedback  of  vehicle  sensor  data  to  the  operator;  and  (C)  shared  control  where  the 
vehicle  does  local  hazard  avoidance  while  the  operator  designates  waypoints  for  the  vehicle  to  follow.  The 
operator  needs  to  make  a  decision  on  which  intelligence  mode  to  choose  to  meet  the  multiple  mission 
objective  while  keeping  the  risk  at  acceptable  levels.  NRMM(I)  generates  data  products  that  assist  the 
operator  in  choosing  the  best  overall  intelligence  level  to  meet  the  scenario  objectives. 


+  Go-No-Go  maps  etc  for 
each  intelligence  mode 


Figure  9-5:  Example  of  the  operational  use  of  NRMM(I)  to  generate  performance/risk  predicts  for  multiple 
autonomy  levels  to  allow  operator  to  select  the  optimal  level  for  carrying  out  the  task. 


Conceptually,  one  way  of  addressing  this  would  be  for  NRMM(I)  to  generate  performance/risk  curves  for 
each  of  the  mission  objectives  for  each  of  the  available  intelligence  modes.  Given  that  there  are  multiple 
objectives,  it  is  likely  that  different  intelligence  mode  options  are  best  suited  for  the  different  objectives. 
Ideally,  the  ranking  of  the  best  intelligence  modes  for  each  objective  is  generated  by  the  NRMM(I)  and  made 
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available  to  the  operator.  Based  on  this  information,  the  operator  can  make  the  final  choice  on  the  intelligence 
mode  to  choose  to  best  meet  the  overall  mission  objectives.  Just  to  complete  the  discussion,  note  that  the 
above  decision  flow  also  applies  to  the  manned  vehicle  NRMM(H)  case  -  except  that  the  multiple  intelligence 
mode  options  need  to  be  replaced  with  the  single  onboard  driver  option. 

In  general,  the  various  vehicle  intelligence  levels  can  be  hierarchical,  or  reflect  a  combination  of  discrete  and 
continuous  settings  within  the  intelligence  modules.  While  vehicle  safety  is  often  given  a  higher  priority  over 
performance,  the  paradigm  described  above  makes  the  safety  and  performance  objectives  explicit,  and  allows 
the  operators  to  make  operational  choices  that  best  meet  the  mission  objectives. 

Thus  it  is  essential  that  NRMM(I)  be  able  to  generate  reliable  quantitative  performance/risk  predicts  for 
available  intelligence  modes  to  support  the  operational  decision  making  described  above.  Such  a  capability 
does  not  currently  exist  for  intelligent  vehicles,  and  requires  sustained  research  and  development  effort  to 
develop.  Without  attempting  to  guess  or  preempt  the  eventual  outcomes  from  such  an  R&D  effort,  we  now 
embark  on  a  potential  approach  to  further  our  thinking  on  the  required  solutions.  A  key  consideration  for  any 
viable  NRMM(I)  solution  is  the  fluid  and  rapidly  evolving  nature  of  vehicle  intelligence.  The  performance  of 
component  intelligence  algorithms  can  be  changed  at  very  little  cost  compared  to  the  costs  involved  in 
changing  vehicle  hardware.  Thus  the  desired  NRMM(I)  solution  needs  to  be  able  to  accommodate  such 
variability  in  generating  predicts.  “One  of’  NRMM(I)  solutions  that  are  brittle  to  such  changes  are  vulnerable 
to  becoming  obsolete  even  before  they  begin  to  see  use. 

With  this  in  mind,  we  explore  a  skills  based  strategy  for  vehicle  intelligence  that  may  provide  an  avenue  for 
the  scalability  required  of  NRMM(I).  From  the  Oxford  dictionary,  “intelligence  is  the  ability  to  acquire  and 
apply  knowledge  and  skills  Based  on  this,  we  propose  the  operational  definition  that  intelligent  vehicles  are 
characterized  by  their  skills  in  executing  vehicle  mobility  tasks  in  a  variety  of  environments.  As  illustrated  in 
the  Figure  9-6  below,  our  notion  of  a  skill  is  rooted  in  the  systems  and  control  ideas  of  modules  that 
implement  a  function  that  processes  inputs  to  generate  desired  outputs,  and  consume  resources  in  the  process. 
It  is  important  to  emphasize  that  a  skill  is  not  a  software/algorithm  attribute,  but  can  also  include  hardware 
resources  for  computing,  sensing,  communication  etc.  Thus  an  autonomous  obstacle  detection  skill  consists  of 
sensor  hardware  for  situational  awareness,  computers  and  memory  to  run  classification  algorithms  to  detect 
obstacles. 
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Figure  9-6:  A  systems  based  representation  of  a  performance/risk  model  for  a  typical  component  skill  illustrating 

its  inputs,  outputs  and  resource  needs. 
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In  the  following  section  we  provide  some  examples  to  illustrate  the  notion  of  skills  for  intelligent  vehicles. 


9.3.2  Examples  of  Intelligent  Vehicle  Skills 

Some  examples  of  capahilities  that  are  pertinent  to  vehicle  intelligence  (and  we  refer  to  as  skills)  are  listed 
helow: 


•  Is  the  vehicle  capable  of  detecting  slippage?  This  ability  means  different  things  for  wheeled  versus 
legged  vehicles.  Is  the  vehicle  capable  of  compensating  for  slippage? 

•  Can  the  vehicle  detect  “instability”  -  vehicle  rollover  for  wheeled,  loss  of  balance  for  legged 
platforms? 

•  Does  the  vehicle  have  situational  awareness?  Under  what  conditions  -  day/night,  clear 
sky/cloudy/rain? 

•  Can  the  vehicle  detect  hazards,  moving  pedestrians  &  other  vehicles,  traffic  lights,  curbs  etc.? 

•  Can  the  vehicle  execute  lane  change  maneuvers? 

•  Can  the  vehicle  follow  traffic  rules? 

•  Can  the  vehicle  do  onboard  path  planning/replanning? 

•  Can  the  vehicle  generate  optimal  options  for  path  to  follow/foot  placement? 

•  Can  the  vehicle  carry  out  coordinated  motion  across  multiple  articulation  degrees  of  freedom  (eg.  for 
manipulation,  legged  vehicle) 

•  Can  the  vehicle  coordinate  mobility  with  manipulation? 

•  Can  the  vehicle  auto-balance,  self-right/recover  for  legged  systems? 

•  Can  the  vehicle  monitor  its  own  health  and  detect  anomalies?  Can  it  autonomously  enter  a  “call 
home”  safe  mode  when  in  trouble? 

•  Can  the  vehicle  learn  from  its  own  current  or  past  success/failure  performance? 

Many  skills  are  hierarchical,  i.e.  higher  level  skill  depends  on  lower  level  skills.  Assisted  driving  features  such 
as  roll  over  stabilization,  distance  following  that  are  increasingly  available  are  examples  of  component  skills 
in  the  intelligence  scale.  As  illustrated  in  Figure  9-7,  the  autonomous  obstacle  avoidance  skill  depends  on 
other  component  skills. 
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Figure  9-7:  Illustration  of  the  hierarchical  nature  of  skills  using  the  obstacle  avoidance  skill  example. 


The  autonomous  obstacle  avoidance  skill  of  an  intelligent  vehicle  in  this  example  depends  hierarchically  on 
the  following  component  skills: 

•  The  obstacle  detection  skill  that  processes  the  lidar’s  sensed  data  to  generate  a  map  of  hazards  in  the 
path  of  the  vehicle. 

•  The  localization  skill  that  uses  onboard  IMU  and  encoder  sensors  to  generate  real-time  estimates  of 
the  vehicle’s  position  and  attitude. 

•  The  terrain  classifier  skill  uses  onboard  maps  together  with  sensed  camera  imagery  to  determine  the 
type  of  terrain  ahead  of  the  vehicle. 

•  The  trajectory  planner  skill  uses  the  current  estimate  of  the  vehicle’s  position  and  attitude  together 
with  the  hazard  map  and  the  type  of  terrain  ahead  to  plan  a  trajectory  that  takes  the  vehicle  towards  its 
goal  while  avoiding  obstacles.  The  planned  trajectory  needs  to  take  into  account  the  characteristics 
(steering,  dynamics,  and  speed)  of  the  vehicle  platform. 

•  The  vehicle  motion  controller  skill  controls  the  steering  and  acceleration  of  the  vehicle  to  follow  the 
trajectory  planned  by  the  trajectory  planner. 

It  is  evident  from  this  example  that  the  overall  performance/risk  characteristics  of  the  obstacle  avoidance  skill 
depends  directly  on  the  performance/risk  characteristics  of  the  underlying  skills.  Thus  the  quality  of  the  IMU 
sensor  affects  the  quality  of  the  vehicle  state  estimate,  while  the  lidar  quality  impacts  the  ability  to  resolve 
hazards.  The  sophistication  of  the  trajectory  planning  algorithm  will  be  reflected  in  the  quality  of  the 
computed  trajectories.  The  motion  control  performance  depends  on  the  number  of  wheels  that  are  steerable,  as 
well  as  the  vehicle  dynamics. 
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9.3.3  Skills  Based  Approach 


The  skill  based  paradigm  allows  us  to  decompose  the  behavior  of  an  intelligent  vehicle  into  a  hierarchy  of 
component  skills,  where  the  performance  of  each  skill  is  limited  to  a  specific  scope  -  and  thus  making  it 
amenable  to  quantitative  characterization  of  its  performance/risk  behavior.  Other  benefits  of  the  skill-based 
approach  are: 

•  Metrics  on  skills  can  be  used  as  a  foundation  for  developing  quantitative  metrics  on  intelligence 
performance. 

•  Mapping  from  skill  metrics  to  higher  level  skill  metrics  though  not  trivial  is  possible  and  may  also  be 
more  computationally  tractable. 

•  Skills  can  be  used  to  support  assessment  of  multiple  intelligence  modes  that  represent  different 
combinations  of  skills. 

•  This  component  skills  approach  allows  expanding  metrics  to  new  types  of  intelligence  modules  as 
they  are  developed. 

•  Understanding  the  sensitivity  of  task  level  performance  on  component  skill  performance  can  provide 
guidance  on  skill  areas  needing  performance  and  risk  improvements. 

•  The  skills  based  paradigm  allows  us  to  focus  on  input/output  behavior  and  be  less  dependent  on 
specifics  of  their  implementation  and  specific  algorithms. 

•  Skills  based  description  of  intelligence  can  also  help  develop  standards  for  intelligence  capabilities 
within  the  community. 


9.3.4  Skill  Performance/Risk  Characterization 

•  Associated  with  each  skill  are  levels  of  performance  and  risk  that  depend  on 

•  vehicle/terrain  dynamics  -  terrain  difficulty  (soil  characteristics,  roughness,  hazards,  slopes) 

•  availability  of  sensor  data  (affected  by  lighting,  fog,  texture,  vegetation,  GPS  availability, 
etc.) 

•  mission  scenario  constraints  &  needs  (e.g.,  time  to  complete,  power,  comm,  bandwidth,  a- 
priori  knowledge  of  terrain,  hostile  or  friendly  terrain) 

•  robustness  to  uncertain  and  unstructured  environments,  anomalies  and  violated  assumptions 
(e.g..  lack  of  texture) 

•  Metrics  reflect  uncertainties  in  inputs,  outputs  and  performance 

•  Shared  control  interactions  that  adjust  skill  level  for  optimal  performance  and  risk 

9.4  NRMM(I)  PRODUCTS 


NRMM(I)  Goals:  The  NRMM(I)  goals  are  in  principle  the  same  as  for  traditional  NRMM,  i.e.  to  generate 
performance/risk  predicts  to  support  assessments  for  vehicle  design  and  operation. 

Intelligence  is  an  additional  layer  over  a  traditional  human  driven  vehicle.  One  of  the  questions  that  arises  is 
the  role  of  the  NRMM(H)  capability  for  manned  vehicles  in  addressing  the  mobility  assessment  requirements 
for  unmanned  intelligent  vehicles. 

•  The  traditional  NRMM(H)  vehicle/terrain  interaction  (VTI)  based  methods  are  based  on  the 
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assumption  that  the  vehicle  control  is  being  carried  out  hy  an  expert  human  driver. 

•  Under  the  assumption  that  the  intelligent,  unmanned  vehicle  will  always  under-perform  the  manned 
vehicle  with  an  expert  driver,  the  GO/NOGO  and  speed  predicts  from  NRMM(H)  can  he  used  as 
hounding,  best  case  predicts  for  the  performance  of  the  intelligent  vehicle.  Under  this  assumption,  the 
no-go  regions  of  operation  predicted  by  NRMM(H)  also  apply  for  intelligent  vehicles  as  well. 

•  The  above  under-performance  assumption  however  is  not  universal  -  because  in  certain  situations  the 
intelligent  vehicle  may  have  superior  performance  since  onboard  intelligence  can  have  more  sensors, 
carry  out  better  sensor  fusion,  have  faster  response,  not  suffer  from  fatigue  and  be  less  prone  to 
sensory  overload  and  distractions. 

NRMM(I)  Products:  During  operations,  the  NRMM(I)  products  need  to  assist  in  selecting  specific  skills 
and  intelligence  modes  that  will  best  meet  the  performance  and  risk  for  the  task  objectives. 

•  GO/NOGO  traversability  maps  &  speed-to-go  are  products  generated  by  NRMM(H)  for  manned 
vehicles 

•  For  intelligent  vehicles,  there  will  be  a  palette  of  available  skill  level  options,  and  for  each  level 
NRRM(I)  needs  to  generate  GO/NOGO  map,  speed-to-go,  performance  metric  predicts  (e.g.,  time  to 
complete,  fuel/energy,  comm  bandwidth,  external  resources)  and  risk  for  the  combination  of  vehicle, 
terrain  and  mission  scenarios  (see  Figure  9-8). 


Intelligence  Model 

■ 

Vehicle  Model 

■ 

Terrain  Model 

\\r/ 


and  Skill/Mode  Options  with 
Perfonnance/Risk  Estimates 


Additional  NRMM(I)  output  for  intelligent  vehicles 

Figure  9-8:  The  expected  output  from  NRMM(I)  consists  of  performance/risk  estimates  for  the  available 

skill/mode  vehicle  mohiUty  options. 


9.4.1  Leveraging  NRMM(H) 

•  For  wheeled/tracked  vehicles  NRMM(H)  mostly  sets  the  performance  ceiling 

•  One  exception  is  drive  comfort  which  may  not  be  a  factor  for  intelligent  vehicles  -  unless 
passengers  are  present. 

•  However,  drive  roughness  can  impact  sensor  and  intelligence  performance  so  it  cannot  be 
ignored 

•  NRMM(H)  may  allow  operator  to  decide  whether  intelligence  is  even  an  option 

•  Are  there  additional  outputs  or  other  requirements  on  NRMM(H)  that  can  be  important  for 
NRMM(I)? 
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•  For  example,  outputs  that  are  pertinent  to  sensors  -  vibration  levels,  occlusions 

•  Terrain  classification  to  include  terrain  properties  (e.g.  adequate  texture)  that  are  important 
for  robust  sensor  performance 

•  Power  consumption 

•  Others? 

9.5  NRMM(I)  PERFORMANCE  MODELS 

NRMM(I)  needs  methods  and  models  that  can  quantitatively  predict  skill  and  system  level  performance  and 
risk  from  vehicle,  terrain  &  mission  specifications.  A  significant  challenge  is  the  large  dimensional  state  space 
of  the  onboard  autonomy  software  and  the  resulting  computational  complexity  for  exploring  and 
characterizing  performance.  We  describe  below  two  approaches  at  opposite  ends  of  the  performance  modeling 
options. 

9.5.1  Black  box/top  down  performance  modeling 

The  black  box  option  does  not  use  knowledge  of  vehicle  intelligence  design  or  implementation.  The  focus  is 
on  characterizing  the  observable  input/output  behavior  of  the  system.  The  black  box  approach  has  been 
pursued  by  the  recent  DARPA  autonomy  grand  challenge  competition’s  for 

•  off  road  driving 

•  urban  driving 

•  humanoid  robotics 

The  DARPA  challenges  designed  specific  test  ranges  and  tasks  to  evaluate  the  system  level  performance  of 
intelligent  vehicles  and  robots,  without  attempting  to  influence  or  evaluate  the  implementation  of  the  systems. 
The  key  to  the  effectiveness  of  the  black  box  approach  is  the  design  of  a  test  suite  that  can  adequately 
characterize  the  performance  of  the  system.  A  real  life  example  of  the  black  box  approach  is  a  driving  license 
test,  where  the  focus  is  not  on  the  how,  and  instead  on  the  evaluation  of  the  licensee’s  skill  under  a  variety  of 
conditions  (e.g.,  test  facility,  obstacle  course,  stress  tests).  The  scores  obtained  on  these  tests  are  used  to  assess 
the  competency  and  skill  level  of  the  driver.  Such  black  box  techniques  are  also  used  for  acceptance  testing  of 
a  new  vehicle. 

•  Pros:  The  black  box  approach  avoids  the  expensive  process  of  understanding  the  system  design  and 
implementation  and  focuses  on  the  direct  evaluation  of  the  system  performance. 

•  Cons:  The  success  of  the  black  box  approach  depends  on  how  well  one  is  able  to  generalize  the  observed 
performance  from  a  limited  number  of  test  conditions  to  real-life  performance  in  the  field.  Considerable 
care  is  required  in  the  design  of  the  depth  and  breadth  of  the  tests  to  provide  adequate  coverage  and  stress 
testing  of  the  system.  A  major  issue  with  the  black  box  approach  is  that  when  the  performance  is  found  to 
fall  short  in  an  area,  the  limited  visibility  into  the  internal  design  makes  it  difficult  to  identify  sub-areas  or 
components  that  need  to  be  improved  to  overcome  the  performance  gap. 

9.5.2  White  box/bottom  up  performance  modeling 

At  the  other  end  of  the  spectrum,  the  white  box  approach  relies  on  a  detailed  knowledge  and  understanding  of 
the  intelligence  layer  architecture  and  design  to  assess  the  performance  of  the  system.  Such  white  box 
techniques  are  also  a  key  aspect  of  system  engineering  processes  that  rely  on  understanding  of  sub-system 
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performance  and  their  cross-coupling  to  carry  out  design  trade-offs  and  improve  overall  system  level 
performance  and  risk.  A  couple  of  examples  of  such  cross-coupling  for  intelligent  vehicles  include: 


•  Sensor  selection  and  placement  on  a  vehicle.  Requirements  include  using  camera  baselines  for 
adequate  resolution,  desired  depth  of  field,  coverage,  low  noise  characteristics,  low -light 
performance,  redundancy,  power/CPU/data  throughput  needs  etc.  The  choices  made  have  a  direct 
impact  on  a  vehicle’s  situational  awareness  and  hence  its  performance 

•  Implementation  of  onboard  motion  control  capability  involves  trade-offs  between  state  update  rates 
(e.g.  via  expensive  visual  odometry  techniques)  and  localization  accuracy.  Trade  choices  have  a  direct 
bearing  on  safe  vehicle  speed  and  robustness,  which  in  turn  affect  system  performance  and  risk. 

The  white  box  approach  decomposes  the  performance  and  risk  assessment  task  into  smaller  performance  and 
risk  assessment  task  for  the  component  modules.  For  instance,  the  vehicle  performance  depends  on  the 
performance  of  the  sensor  suite  in  terms  of  coverage,  sensor  errors,  update  rates,  robustness  under  range  of 
conditions.  Another  example  is  the  impact  that  the  quality  of  state  estimation  layer  as  measured  by  its 
accuracy,  robustness  of  sensor  fusion  etc.  under  range  of  conditions  has  on  higher  level  performance. 
An  understanding  of  the  dependence  of  the  higher  level  performance  and  risk  sensitivity  on  those  of  its 
components  can  provide  a  clear  understanding  on  the  coupling  between  component  and  higher  system  level 
performance  and  risk. 

•  Pros:  The  white  box  approach  provides  detailed  understanding  of  performance  sensitivity  needed  for 
design  changes  and  options  selection  during  operations.  Moreover,  the  decomposition  into  component 
layers  can  help  make  the  evaluation  problem  computationally  tractable 

•  Cons:  Assessment  requires  detailed  understanding  of  internal  design,  and  assessments  are  specific  to  the 
intelligence  architecture 

In  their  purest  form,  the  dual  white  box  and  black  box  approaches  represent  opposite  ends  of  approaches  for 
system  performance  assessment.  They  differ  in  the  level  of  abstraction  used  for  representing  the  system.  In 
practice,  we  should  expect  a  gray  box  approach  to  be  pursued  where  the  level  of  abstraction  is  somewhere  in 
between  the  extremes  of  the  white  and  black  box  approaches.  The  idea  is  to  strike  a  balance  between 
exploiting  knowledge  of  the  intelligence  structure  and  the  complexity  of  characterizing  the  inter-dependency 
between  the  system  and  component  system  performance.  Indeed,  the  skills  based  paradigm  provides  a  way  to 
adjust  the  level  of  abstraction  by  choosing  the  granularity  of  decomposition  used  for  the  skills  hierarchy. 


9.6  NRMM(I)  METHODS,  TOOLS,  BENCHMARKING 

The  development  of  NRMM(I)  will  require  the  advancement  of  modeling  and  simulation  capabilities,  and 
methods,  tools  and  benchmarking  techniques  for  vehicle  performance  and  risk  assessment. 

9.6.1  M&S  Architecture  Needs 

The  NRMM(H)  approach  has  in  the  past  largely  relied  on  empirical  models,  and  is  transitioning  to  a  blend  of 
modeling  and  simulation  (M&S)  techniques  that  rely  on  physics-based  and  semi-analytical  computational 
models.  The  new  capabilities  are  expected  to  be  cost-effective,  computationally  tractable,  and  easier  to 
generalize  and  be  adaptable  to  new  vehicle  and  scenario  needs. 

In  principle,  NRMM(I)  should  be  able  to  build  upon  the  new  NRMM(H)  M&S  capabilities.  The  types  of 
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models  that  would  be  needed  in  such  an  intelligent  vehicle  M&S  capability  would  include  models  and 
modules  for: 

•  Vehicle  dynamics 

•  Sensors 

•  Intelligent  system  algorithms 

•  Environment 

•  Human  cognition  (for  remote  operator) 

While  we  can  expect  to  leverage  mature  capabilities  in  the  vehicle  dynamics  area  from  NRMM(H),  the  other 
areas  are  new  ones  needed  for  NRMM(I).  At  the  minimum,  development  and  test  efforts  are  needed  to 
develop  a  suite  of  validated  high-fidelity  models  in  the  new  areas  for  NRMM(I)  to  build  upon.  Among  the 
challenges  in  developing  such  a  foundational  capability  are: 

•  Validation  of  model  performance  under  the  variety  of  unstructured  and  uncertain  operational 
conditions  for  intelligent  vehicle  operation 

•  Integration  of  models  from  multiple  domains  to  work  together,  and  validation  of  the  integrated  model 
performance 

•  The  large  footprint  and  computational  demands  of  the  models 

Once  such  a  modeling  capability  exists,  in  theory  we  can  exercise  it  over  a  parameter  set  representing  the 
scenario  uncertainty  to  generate  predicts  for  the  system  performance  and  risk.  For  intelligent  systems  such  a 
parameter  set  can  be  expected  to  be  large  for  unstructured  and  uncertain  operational  environments,  with  large 
computational  cost  for  each  run.  So,  while  such  a  suite  of  foundational  models  is  essential,  the  routine  use  of 
such  a  kitchen  sink  simulation  with  high-fidelity  validated  models  all  the  time  will  be  computationally 
prohibitive  and  impractical.  In  practice,  research  and  development  for  advancing  M&S  architectures  is 
required  for 

•  Agile  M&S  architectures  that  allow  the  integration  of  models  from  multiple  domains,  as  well  as 
swapping  them  out  due  to  changes  in  intelligence  sensors,  algorithms,  logic  and  parameters 

•  M&S  architectures  that  allow  the  swapping  out  and/or  idealization  of  scaffolding  models  in  order  to 
focus  on  characterization  of  the  closed-loop  performance,  robustness  and  sensitivities  of  specific  sub¬ 
systems.  Note  that  such  stubbing  out  will  effect  both  hardware  and  their  corresponding  software 
algorithms.  For  instance,  idealizing  the  performance  of  the  localization  algorithm  may  require  the 
replacing  of  the  combination  of  camera  sensor  models  as  well  as  machine  vision  algorithms  with  an 
idealized  virtual  sensor  that  provides  similar  outputs. 

•  M&S  architectures  that  allow  the  use  of  models  at  different  fidelity  levels.  Such  a  capability  can  be 
used  to  trade  off  model  fidelity  for  reduced  computational  cost.  Thus  for  instance  it  may  be 
advantageous  to  use  fast  GPU  hardware  and  algorithms  for  vision  sensor  modeling  instead  of  the 
more  accurate  but  computationally  demanding  ray  tracing  techniques  when  appropriate.  Or  one  may 
choose  to  work  with  idealized  pin-hole  camera  models  instead  of  higher-fidelity  camera  models  that 
handle  non-idealities  such  as  non-square  pixels,  radial  distortion  etc.  However,  such  choices  cannot 
be  made  in  isolation  since  machine  vision  algorithms  rely  on  camera  calibration  parameters,  and  will 
not  perform  as  expected  if  the  hardware  simulation  is  changed  independently.  The  M&S  architecture 
needs  to  allow  the  ability  to  make  fidelity  trades  without  compromising  the  consistency  and  integrity 
of  the  simulation.  An  important  consideration  is  to  avoid  over  interpretation  of  the  results  when  using 
lower-fidelity  models  since  the  range  of  applicability  of  the  results  is  narrower. 


UNCFASSIFIED:  Distribution  Statement  A.  Approved  for  public  release;  distribution  is  unlimited 


66 


Moreover,  even  with  the  existence  of  a  foundation  of  validated  high-fidelity  models  for  intelligent  vehicles, 
their  use  for  kitchen-sink  M&S  on  a  routine  basis  is  impractical  on  a  routine  basis  due  to  the  large 
computational  resources  needed.  We  need  instead  a  process  and  model  flow  such  as  illustrated  in  Figure  9-9. 


Stochastic  & 
parametric 

Scenarios  inputs 


Field  data 


•  Validation 

•  Generalizability 

•  Coverage 


Figure  9-9:  The  model  pipeline  spanning  high-fidelity  vehicle  mohility  models  to  operational  vehicle  performance 

models  needed  hy  NRMM(I). 


This  figure  uses  as  a  starting  point  a  high-fidelity  intelligent  vehicle  M&S  capability  as  described  above  that  is 
capable  of  simulating  desired  scenarios  over  a  suite  of  uncertain  parameters  characterizing  the  environment 
and  operation.  Such  a  capability  is  expected  to  be  computationally  demanding.  The  blocks  on  the  right 
describe  a  pipeline  for  extracting  simpler  performance/risk  models  that  while  less  accurate  are  also 
significantly  less  computationally  demanding.  This  pipeline  assumes  that  the  high-fidelity  M&S  can  be  run 
offline  on  high  performance  computing  platforms  to  simulate  intelligent  vehicles  over  a  large  scenario 
envelop.  The  results  of  these  simulations  would  be  archived  in  a  large  performance  database.  The  database 
data  can  also  be  used  to  store  data  collected  from  intelligent  vehicles  during  field  operations.  The  next  block 
describes  algorithms  and  methods  that  process  the  simulation  and  field  data  to  extract  simpler  surrogate 
models.  While  computationally  simpler,  these  surrogate  models  will  be  of  lower  fidelity  and  with  a  narrower 
range  of  applicability.  Since  the  data  sets  are  large,  this  process  would  be  an  ideal  candidate  for  automation. 
The  last  block  consists  of  a  repository  for  surrogate  models  that  can  be  used  to  predict  intelligent  vehicles 
performance  over  a  variety  conditions.  A  key  requirement  on  the  surrogate  model  repository  will  be  the  extent 
of  coverage  of  the  expected  use  cases,  because  the  individual  models  are  expected  to  have  narrower 
applicability.  Gaps  in  coverage  or  encountered  weaknesses  are  expected  to  be  fed  back  to  the  first  block  to 
trigger  additional  high-fidelity  M&S  runs  and  expansion  of  the  performance  database. 

Such  a  model  pipeline  architecture  will  be  capable  of  meeting  the  varied  and  evolving  capabilities  of 
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intelligent  vehicles.  Moreover,  an  important  advantage  is  that  the  surrogate  models  will  he  based  on  a 
foundation  of  high-fidelity  models.  Currently,  the  component  capahilities  across  the  entire  pipeline  are  not 
available.  At  best,  we  can  currently  find  component  capabilities  that  are  domain  specific  that  would  need  to 
adapted  and  integrated  together  into  such  a  pipeline. 


9.6.2  NRMM(I)  Methods 

Some  of  the  new  methods  needed  for  development  of  NRMM(I)  are  listed  below: 

•  Skill  decomposition  and  skills  taxonomy  for  classes  of  intelligence  vehicles:  The  skill-based 
performance/risk  assessment  approach  requires  the  decomposition  of  higher  level  skills  into 
component  skills.  Techniques  are  needed  to  systematically  define  a  taxonomy  and  a  decomposition 
process.  Clearly  the  skill  set  will  depend  on  the  type  of  vehicle,  environment  and  its  use  and  will  vary 
across  wheeled,  tracked,  in-traffic,  off-road,  indoors,  legged  platforms  etc. 

•  Component  skill  performance/risk  modeling:  Given  a  skill  decomposition,  we  need  methods  to 
quantitatively  assess  their  performance/risk  under  a  variety  of  conditions.  These  techniques  can  be 
combination  of 

o  Analytical  techniques 

o  Simulation,  Monte  Carlo  &  empirical  methods 
o  White/black/gray  box  performance  assessment  methods 

•  Task  level  performance/risk  models  based  on  component  skill  models:  Given  performance/risk 
models  for  component  skills,  we  need  methods  to  combine  these  to  predict  integrated,  higher  level 
performance  and  risk.  Again,  these  may  consist  of 

o  Analytical  techniques 

o  Simulation,  Monte  Carlo  &  empirical  methods 

•  Multiple  levels  of  NRMM(I):  We  need  methods  to  develop  different  levels  of  NRMM(I)  for  use  off¬ 
line  for  detailed  and  accurate  analyses,  as  well  as  ones  that  can  work  under  more  restrictive 
computing  and  time  line  constraints.  Example  options  include: 

o  Off  line,  highest  fidelity  models  (HPC,  cloud  resources) 
o  Workstation  NRMM(I)  for  analyst  and  remote  operator  use 
o  Rapid  response  NRMM(I)  models  for  operational  field  use 

•  Vehicle  dynamics  and  autonomy  performance  coupling:  One  of  the  current  gaps  between  the 
vehicle  dynamics  and  autonomy  communities  is  the  lack  of  systematic  understanding  of  the  coupling 
between  the  two  areas.  These  are  central  to  NRMM(I),  and  as  such  we  need  to  improve  the 
understanding  of  the  relationship  between  them.  This  can  help 

o  improve  combined  NRMM(H)  and  NRMM(I)  coupling  &  capabilities 
o  improve  intelligent  vehicle  and  control  design 

•  Vehicle  dynamics  models:  While  the  dynamics  modeling  of  wheeled  and  tracked  vehicles  has  been  a 
major  research  area,  gaps  remain  for  modeling  vehicle  dynamics  over  soft-soil,  wet  conditions  etc. 
While  NRMM(H)  is  expected  to  invest  in  meeting  these  gaps,  intelligent  vehicles  can  include  non- 
traditional  vehicles  (e.g.  legged,  indoor)  for  which  validated  dynamics  models  remain  sparse. 
Moreover  there  are  also  opportunities  for  leveraging  new  multibody  techniques  (e.g,.  recursive 
methods,  parallel  techniques)  for  improving  computational  speed  and  accuracy  that  are  not  yet  main 
stream  for  the  vehicle  dynamics  community  but  are  widely  used  within  the  robotics  community. 
There  also  remain  open  questions  about  the  applicability  of  accepted  vehicle  terrain  interaction 
techniques  that  have  historically  been  developed  for  large  vehicles  to  the  smaller  platforms  used  for 
intelligent  vehicles.  The  development  of  validated  and  computationally  tractable  models  for 
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intelligent  vehicle  dynamics  simulations  is  a  critical  need  for  NRMM(I). 

•  Intelligent  vehicle  modeling  and  simulation  architectures:  Conventional  modeling  and  simulation 
of  vehicles  has  largely  focused  on  capturing  the  correct  physics  of  the  vehicle  and  vehicle/terrain 
interaction  and  the  M&S  architecture  designs  reflect  this  emphasis  in  the  available  COTS  and  non- 
COTS  toolkits.  While  fidelity  remains  an  important  factor  for  intelligent  vehicles,  additional 
important  factors  for  intelligent  vehicle  M&S  architectures  are  the  ability  to: 

o  include  new  types  of  models  (e.g.  sensors) 

o  integrate  models  and  autonomy  algorithms  from  across  multiple  domains 

o  support  for  stubbing  out  peripheral  sub-systems  in  order  to  focus  performance  analysis  on 
selected  sub-systems 

o  use  models  at  different  levels  of  fidelity  for  non-critical  areas  to  improve  computational 
performance 

o  use  HPC  simulations  for  large  throughput 

•  Extracting  performance/risk  models:  Given  the  large  dimension  of  the  state  space  for  intelligent 
vehicles,  it  is  computationally  impractical  to  rely  entirely  on  high-fidelity  simulations  for  all 
NRMM(I)  performance/risk  assessments.  Methods  to  extract  computationally  tractable  models  from 
available  performance  data  will  go  towards  making  NRMM(I)  practical  in  the  field  or  when  there  are 
time  constraints.  There  is  little  by  way  of  success  stories  to  build  upon  on  this  front,  though  deep 
learning  and  other  machine  learning  technologies  are  highly  relevant  -  especially  for  automating  the 
process.  Another  important  factor  is  for  the  models  to  be  easily  extensible  and  adaptable  to  changes  in 
intelligent  vehicles  and  scenarios  or  as  additional  field  data  becomes  available. 

•  Man/machine  interaction  models:  For  the  foreseeable  future  we  expect  to  see  shared-control 
techniques  to  be  used  for  intelligent  vehicles  with  a  remote  operator  in  the  loop  managing  the  level  of 
autonomy  on  the  remote  vehicle  and  the  operator  console.  Thus  modeling  the  intelligent  vehicle 
effectively  requires  models  for  the  remote  operator’s  behavior  and  interaction  with  the  vehicle.  This 
requires  the  development  of  human  cognition  and  human-machine  interaction  models  that  can  be  used 
for  NRMM(I)  for  intelligent  vehicles. 

•  Relevant  technologies:  Methods  from  other  technical  areas  that  may  be  of  use  for  NRMM(I) 
modeling  include: 

o  Uncertainty  quantification:  The  uncertainty  quantification  area  focuses  on  methods  for 
quantifying  uncertainties  in  model  outputs  and  their  propagation  through  other  models.  These 
methods  are  very  relevant  to  similar  needs  for  the  quantification  and  propagation  of 
performance  and  risk  through  the  skills  hierarchy. 

o  Autonomy  validation  technologies:  While  there  is  extensive  investment  in  the  development  of 
autonomy  technologies,  the  area  of  autonomy  validation  remains  in  a  relatively  nascent  stage. 
However  autonomy  validation  deals  with  the  same  challenges  of  assessing  performance  and 
risk  for  high-dimensional  autonomous  systems  as  NRMM(I)  and  there  is  strong  potential  for 
carryover  of  techniques  across  these  areas. 

o  System  engineering  methods:  An  important  aspect  of  system  engineering  is  the  need  for 
assessing  the  impact  of  and  the  sensitivity  of  overall  system  performance  to  sub-system 
performance  in  order  to  carry  out  system  level  trades.  For  intelligent  vehicles,  there  is  a 
similar  parallel  within  the  hierarchy  of  skills,  where  it  is  desirable  to  understand  the 
sensitivity  of  the  performance/risk  of  a  skill  to  the  performance/risk  changes  of  its  component 
skills. 

•  Alternatives  to  skill  based  paradigm:  While  we  have  devoted  attention  here  to  a  skills  based 
approach  for  characterizing  the  performance  and  risk  of  intelligent  vehicles,  there  are  potentially  other 
approaches  which  may  be  relevant  and  offer  advantages  to  the  NRMM(I)  development  that  should  be 
investigated. 
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9.6.3  NRMM(I)  Tool 


In  contrast  with  NRMM(H)  which  has  many  decades  of  development  and  a  suite  of  capable  tools  to  huild 
upon,  NRMM(I)  is  in  its  infancy,  with  a  lot  of  ground  to  cover  in  methods  and  architecture  development  that 
can  provide  the  ground  work  for  the  development  of  a  tool  suite  for  NRMM(I).  Some  of  the  potential  tools  of 
relevance  to  NRMM(I)  at  this  stage  are: 

•  Closed  loop  dynamics  simulations  with  sensors,  intelligence  algorithms  and  scenarios 

o  Current  M&S  technologies  and  tools  provide  a  good  foundation  on  HPC  and  clouds  for  off¬ 
line,  large  state  space  exploration 

•  Simulation  options  for  workstation  and  field  use  are  quite  limited 

o  Current  options  are  mostly  fragmented  across  autonomy  and  vehicle  dynamics  domains 

o  Need  computationally  tractable  tools  for  intelligence  scenarios  with  adequate  dynamics 
fidelity 

o  Flexible  simulation  tool  architectures  for  isolating  subsystems  to  assess  performance 

•  Machine  learning  tools  and  techniques 


9.6.4  NRMM(I)  Benchmarking 

As  discussed  earlier,  the  white-box  and  black-box  approaches  can  be  regarded  as  opposite  extremes  for  testing 
approaches  used  to  evaluate  the  performance  of  a  system,  while  we  expect  that  in  practice  NRMM(I)  will  use 
a  grey  box  approach  that  lies  somewhere  in  the  middle.  Benchmarking  and  test  areas  needing  development  for 
NRMM(I)  include: 

•  For  the  top-down,  black-box  approach,  effective  performance  assessment  is  dependent  entirely  on  the 
test  sets  and  scenarios  used  to  measure  performance  and  risk.  As  such,  the  benchmark  testbed  suite 
needs  to  include  tests  and  scenarios  of  sufficient  quality,  depth  and  breadth  to  extract  information  that 
provides  sufficient  coverage  and  insight  into  the  system  performance,  and  in  a  way  that  performance 
predicts  can  be  derived  for  real-life  scenarios  that  fall  outside  the  test  suite.  A  challenge  here  is  to 
meet  these  benchmarking  objectives  without  a  large  and  burdensome  test  suite  that  is  expensive  and 
impractical  to  exercise.  Another  important  consideration  for  the  benchmark  suite  is  its  ability  to  adapt 
and  be  extensible  to  changes  to  the  intelligent  vehicle  and  its  usage.  Brittle  and  highly  specialized 
testbeds  will  quickly  become  obsolete  due  to  variability  of  intelligent  vehicles.  The  benchmark  test 
suite  will  need  to  include  a  combination  of  nominal,  as  well  as  (possibly  unrealistic)  stress  tests  to 
help  tease  out  the  knees  in  system  performance. 

•  The  bottoms-up,  white-box  approach  for  performance  assessment  depends  upon  a  detailed 
understanding  of  the  design  and  implementation  of  the  intelligent  vehicle  hardware  and  software.  The 
benchmarking  and  test  needs  for  this  approach  are: 

o  Benchmark  skills  test  suite  to  assess  and  validate  component  skill,  and  sub-system 
performance  and  risk 

o  Benchmark  task-level  test  suite  to  assess  and  validate  task  performance  models 
o  Benchmark  and  test  suites  to  measure  the  sensitivity  of  a  sub-system’s  performance  to 
changes  in  the  performance  of  its  component  sub-systems  throughout  the  system  hierarchy, 
o  Once  again,  the  benchmarking  methods  and  test  suites  developed  here  need  to  be  able  to 
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accommodate  changes  to  the  intelligent  vehicle  software  and  hardware. 

Since  intelligent  vehicles  are  expected  to  operate  in  unstructured  and  uncertain  environments,  the  above 
methods  will  need  to  he  used  within  a  stochastic  testing  framework  to  generate  performance  and  risk 
envelopes.  Techniques  from  design  of  experiments  and  other  sampling  techniques  will  he  invaluable  for 
keeping  the  test  suite  manageable.  An  important  side  benefit  of  the  development  of  such  benchmark  and 
test  suites  is  that  this  might  help  standardize  vendor/provider  designs,  interfaces  and  architectures,  which 
can  have  a  significant  impact  on  the  variability  that  the  testing  framework  needs  to  be  able  to 
accommodate.  Such  a  development  may  also  allow  requirements  to  be  placed  on  vendors  to  provide  skill 
models  for  their  hardware/software  during  the  procurement  process. 

9.7  SUMMARY 

We  summarize  below  key  ideas  pertaining  to  the  development  of  NRMM(I)  for  intelligent  vehicles: 

•  Intelligent  vehicles  still  remain  new  -  though  rapidly  evolving  -  technology,  and  NRMM(I)  has  to  be 
able  to  adapt  and  grow  with  it 

•  We  have  outlined  a  skills  based  framework  for  characterizing  vehicle  intelligence  and  its  many  modes 

•  This  can  form  the  basis  for  quantitative  performance/risk  metrics  that  are  essential  for  NRMM(I)  - 
and  allow  scaling  to  new  classes  of  intelligent  vehicles 

•  Beyond  GO/NOGO  like  data  products,  NRMM(I)  needs  to  provide  assistance  for  selecting 
intelligence  mode  best  suited  for  managing  scenario  performance/risk  during  operations 

•  NRMM(I)  can,  and  should  be  designed  to  build  upon  NRMM(H)  capabilities 

•  Proposed  NRMM(I)  roadmap  is  currently  aspirational,  and  significant  methodology  challenges  need 
to  be  addressed  in  developing  a  quantitative  approach 

•  Maturity  level  is  low,  so  high  priority  to  develop  capabilities  since  intelligent  vehicles  are 
already  being  deployed 

•  Long  road  ahead  to  achieve  NRMM(H)  like  capability  and  maturity 

•  A  concrete  plan  needs  to  be  developed  to  prioritize,  scope  and  make  progress  in  the  near  and 
longer  term 

The  research  was  carried  out  in  part  at  the  Jet  Propulsion  Laboratory,  California  Institute  of  Technology, 
under  a  contract  with  the  National  Aeronautics  and  Space  Administration.  Government  sponsorship  is 
acknowledged. 
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9.8 
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Chapter  10  -  THEME  5:  TOOL  CHOICES 

Henry  Hodges 


10.1  GOALS  AND  DELIVERABLES 

The  Goals  of  Theme  5  are  the  following: 

•  Identify  eritical  elements  for  physics-based  Next  Generation  mobility  model  utilizing  strengths 

and  weakness  criteria  provided  by  initial  “pros  and  cons”  review  of  current  NATO  Reference 
Mobility  Model. 

•  Integrate/coordinate  tool  choice  evaluation  with  other  themes  within  the  overall  effort, 

particularly  Requirements  and  Methodology. 

•  Identify  potential  solutions  throughout  the  technical  community  and  user  nations. 

•  Provide  a  robust  review  process  utilizing  approved  Request  for  Information  (RFI)  and 

Combinatorial  Trade  Study  (CTS)  processes. 

This  summary  report  identifies  the  ability  of  current  and  projected  future  physics-based  simulation 
environments  to  provide  accurate  and  timely  results  which  can  be  used  to  support  vehicle  system 
development,  acquisition,  prediction  of  vehicle  performance  in  an  adverse  operational  environment, 
and  force  projection  metrics  in  the  areas  of  accuracy,  speed,  supportability,  validation,  sustainment, 
and  cost;  and  the  ability  of  physics-based  simulation  tools  to  address  the  current  capabilities  and 
limitations  of  the  existing  NRMM  tool  set. 

The  theme  members  are  shown  below: 


Country 

Name 

Germany 

Gericke,  Rainer 

Germany 

Hoenlinger,  Michael 

Turkey 

Akalin,  Ozgen 

USA 

Gunter,  David 

USA 

Hodges,  Henry:  Leader 

USA 

Jain,  Abhinandan  (Abhi) 

USA 

Jayakumar,  Paramsothy 

USA 

McDonald,  Eric 

USA 

Shoop,  Sally 

USA 

Ward,  Derek 

10.2  TOOL  CHOICE  DESCRIPTIONS 

In  summary  there  are  two  basic  approaches  to  the  prediction  of  vehicle  performance  over  complex  and 
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mobility  challenging  terrain.  There  are  simulation  and  prediction  tools  which  are  based  on  historically 
measured  performance  of  complete  vehicles  and  various  components.  The  relationships  developed 
using  these  field  and  laboratory  measurements  to  generate  algorithms  are  generally  referred  to  as 
empirical.  Then  there  are  simulation  tools  which  are  “physics-based”  and  these  generally  take  all  of 
the  various  terrain  and  vehicle  component  and  system  parameters  and  then  utilize  either  energy 
management  or  equations  of  motion  to  predict  the  performance  of  a  vehicle  system.  There  are  also 
solutions  which  combine  both  empirical  and  physics-based  analysis,  utilizing  empirical  or  look-up 
tables  to  represent  certain  elements  of  the  vehicle  terrain  interaction  and  then  relying  on  the  physics - 
based  tools  to  determine  mobility,  performance,  stability,  and  other  vehicle  system  parameters.  Within 
this  study,  all  potential  solutions  were  considered. 

10.2.1  Questions  to  be  Addressed 

1.  Do  adequate  physics -based  modeling  and  simulation  tools  exist  either  in  the  public  domain  or 
provided  by  industry  which  can  be  used  to  accurately  represent  the  key  mobility  elements  which 
affect  ground  vehicles  and  are  those  tools  currently  affordable  and  implementable? 

2.  What  are  the  key  benefits  of  using  physics-based  modeling  tools  over  empirical  tools  to  tbe  three  end 
users  (operational  planners,  acquisition  officers,  vehicle  designers)? 

3.  How  will  the  NATO  or  other  user-specific  mission  profile  events  be  described  and  provided  to  the 
simulation  environment? 

4.  What  are  the  most  important  capabilities  of  the  existing  NRMM  tool  set  and  what  are  the  greatest 
limitations,  and  how  do  the  various  simulation  solutions  improve  upon  the  existing  tool  set? 

10.2.2  Framework 

The  initial  focus  for  development  of  potential  replacement  tools  was  to  establish  a  framework  through  which 
the  mobility  analysis  tools  could  continue  to  be  updated  and  new  technological  improvements  could  be  added. 
To  that  end,  the  following  framework  statement  was  developed. 

A  ground  vehicle  mobility  modeling  and  simulation  architectural  specification  applicable  to  the  full 
range  of  ground  vehicle  geometric  scales  that  promotes  standardization,  integration,  modular 
interoperability,  portability,  expansion,  verification,  and  validation  of  vehicle-terrain  interaction 
models  at  multiple  levels  of  theoretical  and  numerical  resolution  for  operational  mobility  planning, 
vehicle  acquisition,  and  vehicle  design. 

10.2.3  Notional  Mobility  Software  Tool  Criteria 

In  determining  the  potential  capabilities  for  the  future  tools,  the  following  were  considered  to  be  important 
and  therefore  were  used  to  help  guide  the  development  of  the  request  for  information  and  the  evaluation 
criteria  for  the  various  potential  solutions. 

•  Can  be  used  to  accurately  determine  minimum  ground  vehicle  mobility  performance  over 
representative  world-wide  mission  profile  conditions 

•  Tool  has  sufficient  accuracy  to  support  pre-hardware  engineering  decisions  and  incorporates  the  latest 
technology 

•  Can  be  used  to  rank  order  designs  or  vehicle  systems 
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-  i.e.,  Solution  A  is  superior  to  solution  B  (down-select) 

-  Current  Government  needs  may  require  greater  fidelity  than  historic  comparisons 

-  Accurate  prediction  of  absolute  values  necessary  for  hardware  selection  and  determination  of 
mission  success 

•  Can  he  used  to  establish  critical  design  parameters  during  development 

-  Ground  contact  pressure,  power  to  weight,  tractive  effort,  ride  quality,  maneuverability,  etc. 

•  Can  be  updated  to  include  new  events  that  reflect  current  mobility  challenges  (Afghanistan  versus 
Southeast  Asia  versus  Fulda  Gap). 

10.2.4  Desired  Software  Capabilities 

•  Minimum  Criteria/Constraints: 

-  Fully  3-D,  multibody  dynamics  (MBD)  including  contact  forces 

-  Model  wheeled,  tracked,  and  legged  vehicles  (wheeled  and  tracked  vehicles  are  the 
priority) 

-  Include  electronic  control  systems  to  accurately  represent  suspension  and  drive  train 
hardware  which  optimize  mobility  and  performance(software/hardware  in  the  loop) 

-  Advanced  powertrain  models  allowing  fuel  economy  assessments 

-  Rigid  and  deformable  bodies  and  terrain 

-  Includes  driver  in  the  loop  model 

-  Template  based  (defined  as  the  ability  to  create  subsystems  for  a  given  vehicle  where 
components  can  be  easily  modified  to  reflect  changes  in  technology  and  then  apply  those 
components  directly  to  established  model  without  the  need  to  build  a  new  vehicle  system 
model) 

o  Includes  all  parts,  forces,  constraints,  outputs 
o  Can  be  used  on  multiple  models 
o  Insures  standard  modeling  practices 

o  Templates  include  communicators  to  automatically  connect  and  exchange  data  with  other 
vehicle  subsystems 

o  Template  contains  the  subsystem  topology 

o  By  changing  the  appropriate  data  such  as  mass  properties,  hard  points,  spring  and  damper 
data,  etc.,  the  same  template  can  be  used  on  a  wide  range  of  vehicles 

-  Validation  possible  in  both  time  and  frequency  domain  as  well  as  ability  to  run  design  of 
experiments  (DOE)  iterations  to  identify  dominant  parameters  and  “comers”  in 
performance 

-  Provides  accurate  (in  terms  of  elements  which  impact  mobility)  representations  of  terrain 
and  mobility  events 

-  Allows  terrain  to  be  updated  based  on  environmental  or  mission  requirement  changes 

-  Provides  “deformable”  terrain  elements 

•  Allow  “Layman”  user  to  mn  simulations 

-  Almost  any  code  can  be  used  by  an  “expert”  but  availability  of  experts  limits  ability  of 
the  solution  to  be  more  widely  used  as  intended. 

-  Implementing  GUI,  tools  and  processes  for  layman  use  is  a  significant  task  (Figure  10-1). 
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Figure  10-1.  Graphical  User  Interface  (GUI  Example.  Intent  is  to  allow  a  non-expert  user  to  run 
simulations.  Note:  the  dialog  box  contains  vehicle  specific  fields  for  setting  up  and  running  a  full 
vehicle  simulation.  Underlying  framework  is  desirable  to  be  template-based. 

10.2.5  Vehicle  Terrain  Interaction 

One  of  the  key  elements  for  sueeess  of  any  future  simulation  will  be  the  ability  to  quantify  the  interaetion 
between  the  vehicle  and  the  terrain  over  which  it  travels.  As  such,  models  for  the  tire  or  track  terrain 
interaction  which  can  address  all  combinations  of  soil  type  and  moisture  content  with  a  broad  range  of 
compactions  will  be  critical  to  future  success.  In  the  case  of  the  tire  system,  accurate  representation  of  tire 
spring  and  damping,  cornering  stiffness  and  compliance  under  free  rolling  and  torque  applied  conditions  will 
be  essential.  These  models  will  address  sinkage,  dynamic  tire  deformation,  lug  engagement,  dynamic  slip/ 
sinkage  relationships,  tractive  force  slip,  lateral  force  slip,  and  multi-pass  effects.  The  tire  surface  models 
should  address  discontinuities  within  the  surface  material  and  accurately  predict  the  interactive  force  slip  and 
terrain  deformation  relationship.  The  tools  should  support  validation  in  both  the  complex  field  and  controlled 
laboratory  environment.  The  more  severe  off-road  military  environment  presents  some  unique  challenges 
including: 

•  Military  off-road  tires  with  aspect  ratio  approaching  1  are  highly  nonlinear  and  uniquely  built  to  meet 
the  severe  off-road  duty  cycle. 

•  On-road  tire  models  have  to  be  substantially  tuned  and  adjusted  to  accommodate  deformable  soil 
conditions.  Therefore  simulations  which  may  work  with  uniform  conditions  found  during  traditional 
on-road  maneuvers  may  be  substantially  less  successful  in  the  analysis  of  off -road  events. 

•  Inclusion  of  finite  element  models  of  the  tires  may  initially  be  necessary  to  accurately  represent  the 
tire  soil  interaction.  These  detailed  models  may  be  replaced  by  other  representative  solutions  to  aid  in 
the  simulation  speed  to  insure  that  the  simulation  tool  can  quickly  compare  the  performance  of 
vehicles  or  estimate  mobility  in  real  time  field  situations. 

•  Because  uniform  ground  contact  pressure  is  often  the  key  to  successful  mobility,  the  ability  of  the 
simulation  to  accurately  quantify  these  parameters  may  be  critical  to  accurate  mobility  prediction. 
Available  tools  have  demonstrated  this  ability,  however,  the  integration  of  these  tools  into  a  full 
vehicle  simulation  may  be  a  significant  challenge  and  therefore  must  be  evaluated  through  this 
process. 

•  Unique  simulation  tools  are  required  to  address  the  interaction  between  tracked  vehicle  systems  and 
the  terrain.  Local  high  stress  and  shear  conditions  at  the  track  grouser  to  soil  or  terrain  element  have 
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to  be  considered.  Due  to  requirements  for  this  type  of  analysis,  the  number  of  specialized  tools  may 
prove  to  be  more  limiting.  Further  consideration  will  have  to  be  given  to  a  combination  of  physics- 
based  and  empirically  based  solutions  to  successfully  quantify  tracked  vehicle  to  terrain  interaction. 

•  Tracked  vehicle  turning  in  soft  soil  represents  a  particularly  challenging  simulation  condition. 
Physical  testing  has  demonstrated  that  local  contact  pressure  at  the  road  wheel  to  track  element  can 
significantly  influence  the  mobility  of  the  system.  Therefore,  to  be  successful,  the  fidelity  of  the 
simulation  will  have  to  be  verified  given  the  established  goals  of  this  effort. 

10.2.6  Potential  Sources 

Within  this  theme  effort,  a  range  of  potential  solution  sources  have  been  considered.  Each  potential  source 
has  different  strengths  and  weakness  and  for  each  potential  source,  the  capability  of  the  solution  has  to  be 
quantified. 

The  following  range  of  sources  was  considered. 

•  Government 

•  Commercial 

•  Open  source 

•  Modular  (representing  a  combination  of  various  tools  and  sources) 

The  following  primary  criteria  were  considered  most  important  in  the  evaluation  given  the  established 
constraints. 

•  Accuracy 

•  Sustainability/Flexibility 

•  Template-based 

•  Cost  (acquisition,  implementation,  and  support) 

10.2.7  Scoring  Protocol 

Although  members  of  the  committee  and  representatives  of  other  countries  were  queried,  no  Government- 
based  simulation  code  other  than  the  existing  modifications  to  NRMM  were  identified.  It  was  noted  by 
representatives  of  Canada,  Germany,  and  other  countries  that  other  solutions  had  been  explored  and 
implemented  due  to  the  known  limitations  of  the  current  release  for  NRMM.  However,  no  organization 
indicated  that  there  was  a  tool  which  existed  that  would  meet  all  of  the  goals  established  by  the  committee. 
Ah  representatives  indicated  that  they  were  currently  utilizing  a  mix  of  commercial,  in-house  developed, 
modified  NRMM,  and  other  tools.  Each  organization  indicated  that  improvements  to  the  available 
methodologies  was  required  to  more  accurately  predict  vehicle  performance  in  the  modem  operational 
environment.  If  was  further  recognized  that  funding  for  continued  development  of  these  tools  which  would 
meet  ah  the  objectives  for  next  generation  NRMM  had  been  limited.  Long  term  funding  to  sustain 
Government-based  solutions  was  generally  identified  as  a  limitation  in  the  current  more  austere  conditions. 
Further  although  each  country  identified  internal  structure  to  support  analysis,  this  analysis  was  focused 
specifically  on  the  country’s  own  vehicles  and  requirements  and  not  generally  available  for  broader 
implementation.  As  such  no  specific  “off  the  shelf’  Government  solution  was  identified. 

Potential  open  source  codes  were  discussed.  Although  there  was  awareness  of  multiple  tools,  their  ability  to 
properly  function  to  meet  the  goals  of  the  next  generation  NRMM  was  generally  unknown.  Stability  of  such 
codes  was  generally  identified  as  a  potential  limitation. 
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All  organizations  identified  the  use  of  some  versions  of  commercially  available  tools  to  quantify  and  predict 
vehicle  performance.  The  availability  of  commercial  three  dimensional  (3-D)  physics-based  tools  was  fully 
recognized  along  with  the  significant  investment  to  improve  those  tools  made  by  vehicle  manufacturers 
worldwide.  When  combined  with  the  current  autonomous  vehicle  development  this  investment  was  estimated 
to  be  in  the  billions  of  dollars.  However,  there  was  no  clear  dominant  tool  which  could  support  vehicle 
dynamics  as  well  as  soft  soil  operation. 

Based  on  the  fact  that  no  clear  solution  or  combination  of  solutions  could  be  identified,  the  decision  was  made 
to  send  out  requests  for  information  (RFI)  to  recommended  and  otherwise  known  participants. 
Recommendations  and  identification  of  tools  already  in  use  by  various  Government  organizations  served  to 
help  determine  the  range  of  organizations  that  were  sent  the  RFI.  The  intent  of  the  effort  was  to  identify 
whether  any  robust  solutions  existed  or  if  a  complete  development  effort  was  required  and  hence  significant 
funding  would  have  to  be  established  in  support  of  the  development  of  the  next  generation  mobility  tool. 

The  committee  then  worked  to  develop  a  set  of  criteria  and  appropriate  questions  to  determine  the  capability 
of  existing  tools  from  a  variety  of  Government,  commercial,  and  university  sources.  The  first  step  was  to 
develop  a  series  of  criteria  and  levels  of  importance  for  the  evaluation  to  meet  the  goals  for  the  next 
generation  NRMM  effort.  Capability  often  conflicts  with  cost,  and  speed  of  analysis  conflicts  with  accuracy. 
To  that  end,  the  Measures  of  Effectiveness  (MOE)  and  Measures  of  Performance  (MOP)  were  established  and 
then  weighted  utilizing  the  Combinatorial  Trade  Study  Process.  The  results  of  that  weighting  are  presented 
below.  As  can  be  seen  from  the  table,  the  accuracy  and  flexibility  of  the  simulation  tools  were  identified  as 
the  most  important  aspects  while  cost  and  the  ability  to  update  and  run  unique  NATO  events  were  less 
important. 


Table  10-1  MOE  and  MOP  Weighting. 


MOE 

Weight 

MOP 

MOE 

MOP 

Composite 

Weight 

Accuracy  / 

Robustness 

Physics  based 

Validation  through  measurement 

37.50% 

16.67% 

12.50% 

Supports  time  and  frequency  domain  analysis 

8.33% 

Template  based 

8.33% 

Flexibility 

Wheeled  or  tracked  vehicles 

37.50% 

20.83% 

Automotive  Subsystems 

8.33% 

Cost, 

License 

5.56% 

Maintenance,  and 

Run  Time 

12.50% 

2.78% 

Run  Time 

Training 

4.17% 

NATO  Specific 
Applications 

Supports  unique  terrain  or  mission  definition 
Worldwide  tool  availability  to  approved  sources 

12.50% 

6.94% 

2.78% 

Worldwide  tool  support 

2.78% 

100.00% 

To  properly  gage  the  level  of  capability  for  each  potential  solution,  five  levels  of  satisfaction  were  established: 
unacceptable,  below  threshold,  threshold,  above  threshold,  and  objective.  Based  on  this  set  of  criteria,  the 
REI  document  was  sent  out  with  the  understanding  that  the  responses  would  be  reviewed  and  evaluated 
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accordingly.  For  the  various  levels  a  score  of  zero  (0),  0.5,  0.7,  0.77,  and  0.85  was  applied,  respectively.  For 
each  category,  should  the  response  he  deemed  to  meet  threshold  or  an  acceptable  level  of  capability,  then  a 
score  of  0.7  was  applied.  If  the  response  was  deemed  unacceptable  then  a  score  of  zero  (0)  was  applied. 


Table  10-2.  Accuracy/Robustness  Satisfaction  Levels 


ACCURACY  /  ROBUSTNESS  WEIGHTED  37.50% 


Physics  Based 


16.67% 


Unacceptable 

1)  Fails  to 
incorporate 
force  & 
moment 
relationships 
in  a  physics - 
based 
dynamic 
format 

2)  Unable  to 
represent 
vehicle 
motion  in 
three 

dimensions 
over  time 


Below 
Threshold 

Incorporates 
basic  inertial 
properties  only. 
Unable  to 
represent  system 
in  all  three 
dimensions 
simultaneously. 
Functions  on 
non-deformable 
surfaces  only.  Is 
only  able  to 
manage 

traditional  tire  or 
track  to  surface 
interface. 

Cannot  address 
exterior  vehicle 
to  obstacle  (tree, 
step,  etc.) 
contact. 


Threshold 

Physics-based 
simulation,  but 
is  limited: 

1)  only  rigid 
body  model  (no 
dynamically 
deformable 
bodies  or 
surfaces) 

2)  has 

representation 
of  three 
dimensional 
performance 
over  terrain 
which  can  be 
initially 
represented  as 
non-deformable 
but  for  which 
the  terrain 
parameters, 
(motion 

resistance,  shear 
strength,  etc.) 
can  be 

represented  in  a 
look-up  table 
which  can  then 
be  applied  to  the 
performance 
calculations  of 
the  vehicle. 


Above 
Threshold 

Simulation  can 
accurately 
represent 
varying  levels 
of  sinkage, 
surface 
coefficient, 
etc.  but 
considers  the 
terrain  to  be 
homogeneous 
within  a 
contact 
element. 


Objective 

Captures 
interaction  of  all 
components, 
subsystems,  & 
systems  &  their 
interaction  with 
the  environment 
based  on 
equations  of 
motions,  force  & 
moments, 
temperature, 
pressure, 
acceleration,  etc. 
Allows  system  to 
achieve  point 
contact  with  the 
environment  & 
predicts  the 
results  of  the 
interaction  of  the 
component, 
subsystems  & 
systems  with  the 
environment. 
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ACCURACY  /  ROBUSTNESS  WEIGHTED  37.50% 

Validation 

Through 

Measurement 

12.50% 

Unacceptable 

Below 

Threshold 

Threshold 

Above 

Threshold 

Objective 

No  ability  to 
directly 
compare 
either  through 
time  history  or 
motion  the 
results  of  the 
simulation 
with  the 
results  of  test 

Rudimentary 
ability  to 
correlate 
simulation 
results  with  test 
results. 

Evaluation 
remains  three 
dimensional  but 
only  basic 
inertial  or  center 
of  gravity 
motion  can  be 
correlated. 

Limited  force 

vector 

comparison  is 
possible 

Ability  to  track 
basic 

suspension  and 
powertrain 
relationships. 
Identifies 
motion  of 
suspension  over 
non-deformable 
terrain 

elements.  Can 
determine 
acceleration  and 
force  at  various 
points  within 
the  vehicle 
system  and 
those  results  can 
be  correlated  to 
measured  test 
events  through 
time  history 
comparison. 
Provides  vehicle 
system  gross 
motion  output. 
Includes  all 
steering  and 
powertrain 
functions  but 
does  not  address 
rapidly 
changing 
component 
responses 
including 
limited  slip 
differentials, 
semi  active 
suspension,  etc. 

Capable  of 

addressing 

adaptive, 

semi-active, 

and  fully 

active 

suspensions. 
Able  to 
include  digital 
backbone  and 
integration 
with  control 
algorithms. 
Supports 
vehicle 
sensing  and 
adjustment  to 
terrain  and  is 
able  to  directly 
compare 
simulation 
results  with 
measured 
results  over 
complex 
terrain  events 

Simulation 
includes 
deformable 
terrain  elements, 
provides 
prediction  of  full 
vehicle  system 
terrain  interaction 
including 
dynamic  sinkage 
for  various  soils 
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ACCURACY  /  ROBUSTNESS  WEIGHTED  37.50% 

Supports  Time 
&  Frequency 
Domain 
Analysis 

8.33% 

Unacceptable 

Below 

Threshold 

Threshold 

Above 

Threshold 

Objective 

No  capability 
to  generate 
time  history 
data.  Model 
is  steady  state 
only,  thus 
only  an 
average  speed 
or  pass/fail 
answer  is 
given. 

Generates 
limited  time 
history  data  (i.e., 
vehicle  average 
speed,  but  no 
information  on 
subsystems) 

Generates 
thorough  time 
history  data  and 
movie  files  of 
complete 
system  & 
components. 
Provides  time 
history 

representation 
over  multiple 
terrain 

discontinuities. 
Provides  time 
history  for 
control 

algorithms  and 
application  to 
multiple 
components 
within  the 
vehicle  system. 
Manages 
algorithm  input 
updates  at  the 
rate  of  10  times 
per  second  of 
real  time 
providing 
closed  loop 
control  updates 
at  10  Hz 
resolved 

Offers 
frequency 
domain 
analysis  of  all 
time  history 
data. 

Offers  further 
post-processing 
like  SRS/PVSS, 
durability 
stress/strain  life, 
etc.  Can  support 
flexible  body 
analysis,  can 
manage  the 
frequency 
response  through 
the  suspension  to 
allow  analysis  of 
unique  dynamics 
including 
resonance  and 
traction  hop,  etc. 
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Table  10-3.  Flexibility  Satisfaction  Levels 


FLEXIBILITY  WEIGHTED  37.50% 

Unacceptable 

Below 

Threshold 

Threshold 

Above 

Threshold 

Objective 

Template 

Based 

8.33% 

Only  general 
vebicle 
cbaracteristics 
are  used 
(GVW,  power 
look-up  table, 
gross  tire 
dimensions, 
track 

lengtb/width) 

Systems  can 
be  modeled 
separately,  but 
the  program 
depends  on 
low-level 
coding  or  text 
file  inputs 

Large  systems 
can  be  modeled 
in  a  plug-and- 
play  fashion 

Limited 
subsystems  / 
components  can 
be  modeled  in  a 
plug-and-play 
fashion 

Objective  criteria  - 
provides 
component, 
subsystem,  and 
system  models 
which  can  be 
interconnected  by 
simply  imbedding 
the  component  into 
the  system  model 
and  having  the 
model 

automatically  solve 
the  performance 
over  any  event  and 
provide  an 
immediate 
comparison  of  the 
difference  in 
performance 
between  the  two 

events 

Wheeled  or 

Tracked 

Vehicles 

20.83% 

Does  not  have 
the  capability 
to  model  a 
track/wheel 
off-road.  On¬ 
road  dynamics 
only 

Only  a  crude 
tire  /  super¬ 
element  track 
model  is 
available 

A  detailed  tire  / 
track  model  is 
available,  but 
customization  is 
limited.  Tire 
pressure, 
sidewall 
strength,  lug 
pattern,  track 
design,  etc.  is 
limited 

Detailed  off¬ 
road  tire  model 
(fidelity  similar 
to  FEM).  Track 
model  includes 
physical  design 
for  pins,  shoes, 
bushings,  etc. 

Detailed  off-road 
tire  model  (fidelity 
similar  to  FEM). 
Track  model 
includes  physical 
design  for  pins, 
shoes,  bushings, 
etc. 
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FLEXIBILITY  WEIGHTED  37.50% 

Automotive 

Subsystems 

8.33% 

Unacceptable 

Below 

Threshold 

Threshold 

Above 

Threshold 

Objective 

Unable  to 

create  a 
template  or 
plug-and-play 
approach 
which  allows 
integration  of 
traditional 
powertrain  and 
suspension 
components 

Provides 

ability  to 

integrate 

subsystems 

but  not 

components. 

Allows  plug- 

and-play  with 

subsystems. 

Provides 

limited 

correlation 

with  similar 

hardware  in 

other 

applications 

(i.e., 

commercial 

vocational 

suspensions 

with 

geometric 
modifications 
to  provide 
wheel  travel 
suitable  for 
severe  off¬ 
road 

conditions) 

Provides 
integration  of  all 
automotive 
subsystems  and 
components  to 
include  all 
rotating,  linear, 
and  non-linear 
systems.  Allows 
plug  and  play  for 
validated 
components  and 
provides 
connectivity 
through 
established 
hardware  and 
firmware 
interface  points. 
Provides  basic 
constant  control 
algorithms  (shift 
profile,  adaptive 
suspension, 
central  tire 
inflation  system 
control  for 
differentials,  abs, 
traction  control, 
stability  control, 
electronically 
controlled 
braking 

subsystems  etc.). 
Supports  basic 
co-simulation 

structure 

Supports 

limited 

autonomous 
representation  - 
(collision 
avoidance,  lane 
following  input, 
etc.)  includes 
intelligent 
vehicle  systems, 
closed  loop,  and 
open  loop 
interactive 
control 

throughout  the 

vehicle  system, 

expands 

Functional 

Mock-up 

Interface  (FMI) 

capability 

Supports  full 
autonomous 
operation  based  on 
terrain  and  vehicle 
sensor  inputs, 
includes  all  drive 
types  from 
traditional  fuel 
fired  to  full  electric 
drive  trains, 
provides  full  drive 
by  wire  utilizing 
gig  Ethernet  digital 
backbone 
representation, 
provides  real  time 
updates  to  control 
algorithms  based 
on  sensor  inputs, 
fully  integratable 
through  FMI, 
manages  all 
flexible  body 
interfaces,  manages 
all  non-linear 
component  to 
subsystem  to 
system  interfaces 
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Table  10-4  Cost,  Maintenance,  and  Run  Time  Satisfaction  Levels 


COST,  MAINTANENCE,  AND  RUN  TIME  12.50% 

Unacceptable 

Below 

Threshold 

Threshold 

Above  Threshold 

Objective 

License 

Proprietary 

Expandable 

Open  source  code 

Open  source 

Open  source:  strong 

5.56% 

code,  no 

code  but  only 

Moderate  cost 

code,  non- 

user  support,  long 

potential  to 

through  the 

(less  than  $5000 

restrictive  usage 

term  support  based 

extend 

purchase  of 

per  seat  fee). 

structure  (install 

on  university  or 

capabilities 

modules/add- 

on  unlimited 

application,  long 

without 

ons,  but  can  be 

machines) 

term  funding. 

Vendor’s 

had  for  a  lower 

extensive  user 

planned  updates. 

involvement 

price 

groups  and 

models  can  be 

support. 

exported  into  any 

deployed  to 

environment. 

more  than  5,000 

Vendor  supported. 

users,  regular 

significant  market 

international  user 

penetration. 

group  meetings. 

integration  with 

broad  application 

multiple  platforms 

beyond 

and  multiple 

automotive 

software  codes,  no- 

utilizing  physics- 

cost  single  user 

based  analysis 

license  for 

simulation-based 

acquisition. 

Run¬ 

Time 

Can’t  run  in 

Runs  in 

Can  run  in  parallel 

Can  run  in 

Can  conduct  real- 

parallel,  does 

parallel  with 

with  up  to  16 

parallel  with 

time  calculations. 

2.78% 

not  work  on 

increased  core 

cores,  works  on 

unlimited  cores. 

while  running  an 

Windows  and 

capability. 

Linux  and 

works  on  Linux 

unlimited  number  of 

Linux 

works  on  at 

Windows  based 

and  Windows 

cores  and  works 

least  Windows 

computers 

based  systems 

with  Linux  and 

based  systems 

Windows  based 

or  Linux 
systems 

computers 
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COST,  MAINTANENCE,  AND  RUN  TIME  12.50% 

Unacceptable 

Below 

Threshold 

Threshold 

Above  Threshold 

Objective 

Training 

4.17% 

No  training 
available. 
Limited  and 
inexperienced 
user  base.  No 
technical 
manuals  or 
published  case 
studies. 

Web-based 
support  and 
tutorials  at 
additional  cost, 
infrequent  user 
group 
meetings, 
limited  market 
penetration, 
limited 
consultant 
support 

Full  web-based 
tutorials  and 
support. 

Troubleshooting 
hotline,  regional 
offices,  yearly 
conferences,  and 
specialized 
training  offered, 
extended 
consultant  base, 
university  support. 
Provide  basic 
novice 

applications  but 
requires  greater 
expertise  to  run 
successfully 

Full  web-based 
tutorials  and 
support. 

Troubleshooting 

hotline,  regional 

offices,  yearly 

conferences,  and 

specialized 

training  offered, 

extended 

consultant  base, 

university 

support. 

Government 

support  provide 

full  expert 

development 

environment. 

Provides  user 

groups 

interaction 

allowing 

implementation 

of  latest  expert 

applications 

Extensive  training 
and  support.  Wide 
and  experienced 
user  base  with  active 
group  meetings  and 
wealth  of  published 
documents. 

Detailed  User’s 
Manuals  are 
required.  Video 
tutorials,  tools 
embedded  in 
university 
environment  and 
included  in 
advanced  degree 
programs, 
conferences  and 
well  established  user 
groups,  modular 
development  with 
outreach  to  other 
disciplines.  Fully 
interactive  with 
established 
mechanical 
engineering, 
autonomous  system, 
structural 
engineering,  etc. 
Novice  and  expert 
development 
capability 
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Table  10-5.  NATO  Specific  Applications  Satisfaction  Levels 


NATO  SPECIFIC  APPLICATIONS  12.50% 

Supports 
Unique 
Terrain  or 
Mission 
Definition 

6.94% 

Unacceptable 

Below 

Threshold 

Threshold 

Above 

Threshold 

Objective 

Variable 
terrain  is  not 
possible  - 
simulation  can 
handle  only 
homogenous 
surface. 

Terrain  is 
defined  as  a  2- 
D  road. 

Terrain  is  not 
considered 
deformable. 
Effects  of 
climate  not 
considered. 

Terrain  is  3-D, 
but  not 
customizable. 
Limited  soft-soil 
effect  available 
(e.g., 

homogeneous 
soft  soil,  but  not 
variable) 

3-D 

customizable 
terrain  that 
supports 
heterogeneous 
soil  conditions 
is  possible,  but 
must  be 
explicitly 
defined.  Cannot 
be  integrated 
with  climate 
conditions. 
Outside  data  can 
be  imported. 

3-D  customizable 
terrain  that  supports 
heterogeneous  soil 
conditions.  Outside 
data  can  be 
imported.  Surface 
conditions  can  be 
altered  depending 
on  climate 
conditions. 

Worldwide 

Tool 

Availability 
to  Approved 
Sources 

2.78% 

Poor 

deployment, 
limited  user 
base,  single 
university  or 
venue  only,  no 
user  groups 

Specialized 
deployment, 
applicable  to 
unique 
requirements 
and  analysis, 
deployed  for 
specific 
markets  such 
as  oil  field, 
unique 
military, 
deployed  to 
support  single 
vehicle  sets 
(i.e.,  captive 
to  a  single 
manufacturer 
such  as  CAT 
or  Renault  or 
Mercedes, 
etc.)  Captive 
to  a  specific 
government 
agency 

Unique  NATO 
events  firewalled 
and  isolated  from 
other  analysis 
within  the 
simulation 
environment  as 
may  be  required. 
Tool  supports 
regular  updates  as 
may  be  designed 
by  NATO  for 
new  events. 
Updates  deployed 
within  30  days 
after  validation. 

Improved 

update 

deployment 

timing 

Immediate  updates 
for  NATO  events  as 
developed.  Regular 
updates  for  NATO 
identified  terrain 
and  mobility 
criteria.  Support  to 
NATO  established 
proving  ground  and 
other  validation  test 

events. 

Environmental 
updates  possible  as 
identified 
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NATO  SPECIFIC  APPLICATIONS  12.50% 

Unacceptable 

Below 

Threshold 

Threshold 

Above 

Threshold 

Objective 

World  Wide 
Tool 

Support 

2.78% 

Little  or  no 
support. 

Single  country 
footprint  of 
sponsor. 

Support  is 
available  only 
through  e- 
mail  or 

telephone.  No 
established 
user  groups. 

Support  provided 
in  all  NATO 
countries 

Support 
provided  to  all 
NATO 

countries,  user 

groups 

established 

through  primary 

technical 

societies 

including 

ASME,  ISO, 

SAE,  Imech, 
etc.  Deployed 
to  multiple 
commercial  and 
government 
agencies, 
extended 
consultant  base, 
integrated  with 
terrain  mapping 
user  groups 

Supporting  entity 
has  a  global 
presence  with 
representation  in  all 
NATO  countries 
and  worldwide, 
deployed  across 
multiple 
disciplines, 
worldwide  on-site 
support,  agreements 
in  place  with 
multiple  specialty 
software  solutions, 
demonstrated 
integration  and 
problem  solving 

10.3  REQUEST  FOR  INFORMATION  (RFI) 


The  purpose  of  the  Request  for  Information  is  to  determine  the  availability  of  sueh  tools  and  to  establish  a 
sustainable  simulation  environment  which  has  the  flexibility  to  incorporate  new  simulation  solutions  as  they 
are  developed.  It  is  further  noted  that  continuing  and  new  research  development  are  necessary  in  specific 
technology  areas.  As  such  a  “template”  based  simulation  environment  is  envisioned  under  the  following 
charter.  The  framework  is  a  ground  vehicle  mobility  modeling  and  simulation  architectural  specification 
applicable  to  the  full  range  of  ground  vehicle  geometric  scales  that  promotes  standardization,  integration, 
modular  interoperability,  portability,  expansion,  verification,  and  validation  of  vehicle-terrain  interaction 
models  at  multiple  levels  of  theoretical  and  numerical  resolution. 

Physics-based  simulation  environments  are  currently  available  either  commercially,  open  source, 
academically,  or  within  Government  agencies.  New  simulation  environments  are  being  developed 
specifically  to  support  current  challenges  from  man-machine  interface  to  complete  vehicle  autonomy.  The 
vision  of  the  RFI  is  to  collect  and  use  available  information  for  the  physics-based  vehicle  and  the  environment 
in  which  that  vehicle  operates  to  establish  the  criteria  for  the  framework  and  to  conduct  a  down-select  with  the 
outcome  being  a  recommendation  for  a  successful  framework  that  would  be  available  for  implementation 
throughout  the  NATO  member  countries  within  3  years. 

The  RFI  seeks  information  specific  to  ground  vehicle  dynamics  simulation,  terrain  mapping,  and  autonomy 
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capabilities.  The  RFI  as  sent  is  found  within  Appendix  E  of  this  report  section  and  includes  six  different 
attachments  as  noted  in  the  RFI. 

The  RFI  included  a  significant  amount  of  information,  identifying  the  intended  details  of  the  vehicle  operating 
environment,  summarizing  the  amount  of  vehicle  data  which  is  considered  to  be  a  minimum  (based  on  the 
current  input  to  the  NRMM),  and  identifying  current  and  future  capabilities  of  interest  along  with  requested 
information  on  cost  and  international  support  capability. 

Initial  discussions  with  Government,  Universities,  and  Industry  indicated  that  appropriate  flexible  multibody 
dynamics  (MBD)  analysis  tools  do  exist  and  are  supported  throughout  the  analysis  community.  Based  on 
that,  as  provided  in  the  RFI,  descriptions  of  capabilities  in  the  following  areas  were  requested. 

•  Integration  of  various  component  modules  into  a  complete  simulation  environment 

•  Use  of  standard  vehicle  terminology,  component  description,  and  vehicle  related  component  interface 

•  A  vehicle  representative  graphical  user  interface  (GUI)  instead  of  individual  detailed  descriptors 

•  Ability  to  customize  vehicle  system  representation  to  reflect  future  vehicle  technologies 

•  Description  of  physics-based  dynamics  for  systems  other  than  traditional  ground  vehicles  (e.g.,  rail, 
air  vehicle,  water  craft,  etc.) 

•  Description  of  the  ability,  should  it  exist,  to  run  current  NRMM  events  and  then  to  supplement  those 
events  with  more  detailed  terrain  elements  including  expanded  description  of  water  to  land  transition 
(bank  or  beach  transition)  and  urban  environment  events  (e.g.,  steps,  rubble  piles,  etc.) 

•  Explanation  of  basic  and  expert  user  environments 

•  Ability  to  lock  and  track  vehicle  component  configurations  which  can  be  correlated  to  detailed 
vehicle  drawing  packages  or  existing  finite  element  models 

•  Database  hierarchy  to  track  and  store  all  vehicle  parameter  references 

•  Ability  to  share  detailed  vehicle  component  data  between  users 

•  Post  processing  capability  to  perform  evaluation  of  model  fidelity  or  to  quantify  the  impact  of  specific 
components  on  overall  vehicle  performance  (Design  of  Experiment) 

As  noted  in  the  prioritization  of  the  key  elements,  the  ability  of  the  physics-based  MBD  analysis  tools  to 
provide  modularity  is  a  key  to  success.  A  modular  approach  to  the  simulation  potentially  saves  time  in 
development,  allows  more  rapid  comparison  of  the  impact  and  various  components,  and  allows  introduction 
of  unique  mission-based  events  without  the  need  to  build  a  completely  new  simulation.  As  noted  within  the 
RFI  additional  detail  on  the  approach  to  modularity,  how  the  various  vehicle  elements  are  connected  (hard 
points,  control  algorithms,  etc.)  is  an  important  part  of  the  evaluation  of  the  potential  capability  of  the 
solution.  Further  the  ability  to  support  future  analytical  solutions  (FEM,  DEM,  terrain  elements,  etc.)  is  a  key 
aspect  to  rating  the  capabilities  of  the  simulations. 

Within  the  RFI,  information  and  examples  of  how  well  the  simulation  correlated  to  both  on-road  and  off-road 
events  were  requested.  Accuracy  and  validation  to  measured  component  and  system  data  are  essential  to  the 
success  of  a  next  generation  NRMM  simulation.  The  approach  to  highly  non-linear  elements  whether  tire, 
suspension,  or  soil  conditions  and  the  validation  against  measured  data  is  essential  information.  As  noted  in 
Appendix  E,  the  desire  is  to  insure  best  accuracy  and  flexibility  to  insure  that  the  solution  can  support  multiple 
platforms  and  future  technologies.  Cost  and  sustainment  of  the  tools  is  also  critical  as  significant  investment 
will  be  made  for  successful  implementation.  The  ability  to  support  the  tools  worldwide  and  support  unique 
NATO  related  events  is  also  explored.  Availability  of  training  both  on  line  and  through  technical  meetings  is 
addressed  with  the  RFI. 
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It  is  recognized  that  probably  no  single  code  will  be  perfect  for  all  objectives.  However  within  the  parameters 
set  by  this  committee  the  desire  is  to  identify  tools  which  can  meet  the  intended  structural  criteria  for 
performance,  validation,  and  future  development.  As  such,  information  was  requested  from  University, 
Government  and  Commercial  entities  as  noted  below.  Discussions  continued  with  the  various  organizations 
with  the  understanding  that  the  responses  would  be  appropriately  scored  and  evaluated.  That  information 
would  serve  to  inform  the  committee  and  appropriately  inform  the  next  step  efforts. 

10.3.1  Next  Steps 

The  RFI  was  developed,  reviewed  by  the  committee,  and  sent  out  to  more  than  40  organizations.  The  RFI 
specifically  addressed  and  requested  additional  information  on  the  ground  vehicle  dynamics  simulation 
environment,  the  structure  of  the  simulation  environment,  and  the  core  basis  for  the  tool  development 
(physics-based,  empirically  based,  modular,  tool  combination,  etc.).  Scoring  criteria  and  prioritization  of 
capabilities  were  provided  and  detailed  information  on  the  user  environment,  training,  control  algorithms,  and 
description  of  interface  with  deformable  terrains  was  requested. 

Specific  detailed  information  was  also  provided  through  Attachments  as  shown  in  the  RFI  provided  in 
Appendix  E.  This  included  specific  information  on  terrain  roughness,  the  use  of  Wave  Number  Spectra 
defined  three-dimensional  terrain  profiles,  outline  of  minimum  data  as  required  by  the  existing  NRMM, 
anticipated  minimum  physics-based  model  input  requirements,  specifics  on  vehicle  dynamics,  details  on 
terrain  mapping  capability  and  the  ability  to  integrate  the  terrain  mapping  with  the  vehicle  simulation 
environment  as  a  single  tool,  and  finally  information  was  requested  on  the  ability  of  the  simulation 
environment  to  include  sensors,  control  algorithms,  and  other  critical  parametric  elements  as  would  be 
anticipated  for  accurately  predicting  autonomous  vehicle  performance. 

The  conceptual  Duty  Cycle  /  Mission  Profile  included  detailed  information  on  the  following  characteristics 
and  requested  information  on  how  the  existing  simulation  solution  would  address  these  various  terrain  criteria. 

•  Primary  Roads 

-  High  quality  to  highly  degraded  pavement 

•  Secondary  Roads 

-  Loose  surface  to  washboard  to  Belgian  Block 

•  Trails 

-  One  lane,  loose  unimproved  road 

•  Cross-Country  Terrain 

-  No  road  or  trail  exists 

The  minimum  data  input  requirements  identified  the  typical  parameters  found  in  the  existing  NRMM  data 
input  sheets.  This  includes: 

•  Typical  parameters  for  interface  between  vehicle  and  environment  (e.g.,  tire/track  and  soil) 

•  Wheel  (or  road  wheel)  and  chassis  characteristics 

•  Unique  info  for  tracked  vehicles 

•  Hull  geometry 

•  Powertrain 

•  Aerodynamics  characteristics  as  applied  to  the  vehicle  configuration 

•  Maximum  braking  coefficient 

•  Swim  parameters  as  might  be  applied  to  a  vehicle  which  can  both  swim  and  transition  to  landward 
operation 

•  Suspension  design  and  characteristics 
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•  Chassis 

•  Steering 

•  General  vehicle  characteristics 

The  generalized  physics-hased  simulation  data  and  vehicle  input  configuration  requirements  requested 
information  on  how  the  simulation  environment  would  address: 

•  Generalized  data  input  for  powertrain  model  and  how  it  might  he  made  modular  with  the  ability  to 
interface  different  transmission  or  torque  converter  configurations  along  with  the  ability  to  address 
new  technology  infinitely  variable  transmission  designs.  The  solution  must  include: 

-  Engine,  torque  converter,  transmission,  transfer  case,  etc. 

•  Generalized  data  input  for  suspension  model  template 

-  Mounting  hard  points,  mass  properties,  bushings,  motion  control,  etc. 

•  Generalized  data  input  for  tire  model  template 

-  Geometry,  mass,  stiffness,  etc. 

•  Generalized  data  input  for  track  model  template 

-  Geometry,  mass,  stiffness,  etc. 

•  Soil  properties 

For  the  vehicle  dynamics  modeling  element  more  than  25  questions  were  posed.  Examples  of  these  inquiries 
include: 

•  Does  the  solver  support  parallel  processing  and/or  other  high  performance  computing  environment? 
If  so,  how  well  does  the  solution  time  scale  when  going  from  2  to  1,000  cores?  Does  the  software  run 
on  both  Windows  and  Linux? 

•  Does  the  model  support  a  template -based  approach?  If  so,  describe  how  this  is  implemented.  What  is 
included  in  a  template?  How  are  the  templates  created  and  modified? 

•  Can  the  tire -terrain  or  track-terrain  contact  support  FEA/DEM  for  deformable  terrain  at  the  contact 
patch/nodes? 

•  Describe  the  level  of  detail  included  in  the  powertrain  and  driveline  model. 

•  How  does  your  software  support  evaluation  of  uncertainty  in  model  parameters?  Are  stochastic 
methodologies  built  in?  Are  capabilities  for  design  of  experiment  (DOE)  included?  Describe  the 
capabilities. 

For  the  terrain  mapping  information  in  addition  to  critical  soil  structure  representation  the  following  questions 
are  typical  of  the  level  of  detail  requested. 

•  Identify  the  types  of  terrain  data  used  in  the  simulation,  and  the  areal  extent  to  be  provided  along  with 
its  precision  and  fidelity. 

•  Are  the  data  supported  in  a  wide  range  of  database  engines,  e.g.,  Microsoft  SQL  Server,  Oracle,  IBM 
DB2,  IBM  Informix,  Interbase,  Firebird,  Sybase,  PostgreSQL,  SQLite,  MSJET,  etc.? 

•  Will  the  data/process  support  import/export  from/to  modeling  and  simulation  software  platforms? 
Describe. 

•  Are  the  data  capable  of  supporting  wide  ranges  of  coordinate  systems  and  projections  for  on-the-fly 
projection? 

•  Are  the  process/data  OGC  compliant? 

In  an  effort  to  address  future  vehicle  system  development,  beyond  traditional  wheeled  and  tracked  vehicles, 
inquiries  were  made  as  to  the  ability  to  address  autonomous  vehicle  systems.  Autonomous  vehicles  require 
unique  tool  capabilities  because  of  their  reliance  on  unique  sensor  technologies  for  successful  operation. 
Challenges  such  as  glass-to-glass  latency,  interaction  of  digital  backbone  elements,  target  recognition  and 
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processing  time,  etc.  can  all  influence  the  ability  of  an  autonomous  vehicle  to  successfully  transit  a  mobility 
event.  Therefore  sample  questions  include: 

•  Can  the  simulation  environment  present  scene-based  operations  which  include  the  challenges 
associated  with  lit  and  unlit  conditions?  Can  the  environment  in  the  simulation  be  impacted  by  fog  or 
dust  or  other  environmental  conditions  which  can  impact  sensor  performance?  Can  it  be  able  to 
control  lighting,  fog  (that  can  affect  sensing)? 

•  How  are  vision-based  sensors  represented,  and  what  are  the  metrics  for  performance?  GPU 
acceleration?  Ray  tracing? 

•  Can  reflectance  properties  (e.g.,  BRDF)  be  specified  for  objects  needed  by  sensor  models? 

•  Is  there  support  for  modeling  interiors  of  buildings  for  indoor  mobility  evaluation? 

Approximately  three  months  was  made  available  for  the  various  sources  to  provide  responses.  Additional 
questions  and  discussions  were  held  throughout. 


10.4  RFI  DISTRIBUTION 

The  RFI  was  sent  to  the  following  companies. 


Response  Received 

No  Response  Received 

Motion  Port 

System  Level  Simulation,  Vi-Grade 

MSC  Software  Corporation 

Virginia  Tech 

Real  Time  Technologies 

Mississippi  State 

University  of  Madison,  Wisconsin 

Comet  Solutions 

CM  Labs 

Mathworks 

Modelon/Xogeny 

Lockheed  Martin 

Vehicle  Simulation  Development  Corp 

Northrop  Grumman 

Advanced  Science  &  Automation  Corp. 

ESRI,  Inc. 

Quantum  Signal 

Clark  Labs 

JPL 

Hexagon  Geospatial 

LMS/Siemens 

Pitney  Bowes 

PTC 

TatukGIS 

SIMPACK  USA 

Google.  Inc. 

Altair 

10.5  SCORING 

As  the  RFI  responses  were  received  from  industry,  each  was  reviewed  for  content  and  accuracy  of  the  various 
questions.  If  answers  provided  were  vague  or  non-committal,  an  email  request  for  clarification  was  submitted 
to  the  organization.  All  subsequent  replies  were  added  to  the  correct  organization’s  RFI  response  file.  The 
four  Measures  of  Effectiveness  were  scored  using  the  Measures  of  Performance  metrics.  Each  metric  utilized 
answers  from  the  REI  responses  that  were  then  scored  against  the  satisfaction  level  criteria  listed  in  Table  10- 
2  through  10-5.  This  would  result  in  a  numeric  satisfaction  level  score  being  assigned  to  that  MOP  metric. 
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The  scoring  varied  from  0.0  to  0.85  using  five  discrete  levels  which  help  delineate  the  various  solutions  that 
were  scored.  The  score  for  each  MOP  typically  consisted  of  two  or  three  metrics  that  were  combined  for  the 
final  score  on  that  specific  MOP.  If  a  0.0  was  received,  the  solution  was  deemed  Unacceptable,  the  content  of 
the  answer  was  vague,  misleading,  non-existent,  or  the  solution  showed  little  or  no  value  to  the  metric.  If  the 
answer  addressed  the  question  but  the  solution  only  showed  partial  ability  or  capability  of  metric  it  was 
awarded  a  Below  Threshold  value  of  0.5.  Solutions  that  fit  the  criteria  for  the  MOP  but  did  not  fully  support 
the  requirement  were  awarded  a  Threshold  value  of  0.7.  An  Above  Threshold  score  of  0.77  was  awarded  to 
those  solutions  that  showed  the  ability  to  meet  or  support  the  capability  required  for  the  MOP.  Finally,  if  the 
solution  met  in  full  or  exceeded  the  capabilities  of  the  MOP,  the  solution  was  awarded  an  Objective  score  of 
0.85.  A  breakdown  of  the  scoring  criteria  is  listed  in  Table  10-6.  The  following  sections  describe  in  greater 
detail  the  MOEs  of  Accuracy,  Flexibility,  Cost,  and  NATO  Specific  Requirements  with  associated  MOPs  and 
scoring  rationale  for  each. 

As  each  of  the  RFI  responses  were  received,  further  information  was  required  to  fully  vet  the  information 
being  provided.  As  a  result,  a  second  round  of  questioning  was  performed  to  gain  further  elaboration.  Those 
answers  were  scored  on  an  informational  basis  thereby  foregoing  the  Unacceptable  through  Objective  levels 
of  satisfaction  and  using  an  A  through  D  scale  to  avoid  any  confusion  in  the  scoring  process.  Those  results  are 
listed  in  Table  10-20  and  10-19. 


Table  10-6.  Scoring  Values 


Objective 

0.85 

Above  Threshold 

0.77 

Threshold 

0.7 

Below  Threshold 

0.5 

Unacceptable 

0. 

10.5.1  Scoring.  Measure  of  Effectiveness:  Accuracy  /  Robustness 

The  MOE  Accuracy  had  three  measures  of  performance  that  were  scored  using  REI  feedback  from  the 
vendors.  The  MOPs  reviewed  were  Physics  Based,  Validation  Through  Measurement,  and  Supports  Time 
and  Erequency  Domain  Analysis. 

Physics  Based  attributes  were  scored  according  to  the  software’s  ability  to  accurately  use  first  principles  of 
physics  to  represent  a  vehicle  and  its  interaction  with  the  environment.  The  vehicle,  its  components,  and  the 
environment  can  be  represented  as  flexible  bodies.  A  high-fidelity  soft  surface  model  is  available.  Variations 
in  terrain  composition  and  related  characteristics  are  modeled  -  the  soil  can  be  modeled  as  a  heterogeneous 
mixture  of  different  soil  particles  with  large  rock  or  void  inclusions. 

In  the  following  tables,  the  software  developers  are  listed  as  Organization  A  to  L  representing  the  twelve 
companies  that  responded  to  the  questionnaire,  for  the  same  of  anonymity. 
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TabIelO-7.  Accuracy  -  Physics  Based 


Accuracy  -  Physics  Based 


Organization 

A 

Threshold 

The  software  met  Threshold  because  it  can  import  geometry  and  actively  link  to  a 
related  CAD  program.  It  appears  to  be  capable  of  flexible  body  modeling,  NVH 
analysis,  and  Eigen  mode  analysis.  It  can  link  to  outside  FEA  software,  or  use  its  own 
engine.  It  did  not  get  ""Above  Threshold"  because  soil  deformation  is  still  under 
development. 

Organization 

B 

Objective 

The  software  met  Objective  because  it  has  integrated  3-D  modeling  for  suspension  hard 
points  and  necessary  vehicle  geometry  for  contact  modeling.  The  software  models 
deformable  bodies  using  finite  elements,  and  is  capable  of  non-linear  deformation  due  to 
geometry  or  materials.  Eurther  two  types  of  tire  models  are  available:  (1)  a  detailed 
finite  element  tire  model;  and  (2)  a  lumped  distributed  contact  polygonal.  Both  models 
are  valid  for  large  vehicle  speeds  and  excitation  frequencies.  Tire-  or  track-terrain 
contact  support  DEM  for  deformable  terrain  at  the  contact  patch/nodes.  An  FEA  terrain 
can  also  be  modeled,  but  is  not  good  on  soft  soil. 

Organization 

C 

Below 

Threshold 

The  software  scored  Below  Threshold  because  it  can  import  CAD  models.  While  the 
main  soil  interaction  is  calculated  with  Bekker-Wong-Reece  terramechanics,  a  hybrid 
particle-surface  model  is  used  for  earthmoving  simulations  -  this  could  be  useful  if 
extended  to  vehicle  mobility.  It  did  not  reach  Threshold  because  deformable  bodies  at 
the  component  level  do  not  appear  to  be  possible.  Also  bodies  are  described  as  lumped 
masses,  thus  stiffness,  damping,  and  friction  characteristics  cannot  vary.  There  were  no 
provisions  for  EE  A  or  DEM  analysis. 

Organization 

D 

Below 

Threshold 

The  software  scored  Below  Threshold  because  CAD  data  can  be  imported  for  both  the 
vehicle  and  environmental  features.  The  software  currently  supports  a  modal  approach 
for  flexible  multibody  dynamics,  but  there  is  no  internal  DEM/EEA  solver.  Co¬ 
simulation  is  possible,  and  would  be  necessary  for  detailed  analysis.  The  environment 
is  modeled  with  a  grid  mesh,  but  only  Bekker-Wong  terramechanics  are  included. 
Different  layers  of  soil  are  possible,  but  they  are  all  assumed  to  be  homogeneous. 

Organization 

E 

Threshold 

The  software  was  scored  Threshold  because  modal  bodies  can  be  imported  for  complex 
part  geometries.  A  program  extension  can  be  used  to  solve  part  behaviors  internally. 
The  software  can  work  with  flexible  bodies  internally,  but  it  isn’t  clear  how  it  handles 
contact.  The  software  heavily  stresses  its  EMI  capabilities,  so  linking  to  external  EEA  / 
DEM  solvers  should  be  able  to  handle  internal  shortcomings.  The  software  does  not 
include  a  detailed  off-road  tire  model,  but  it  can  interface  with  ETire  which  includes 
both  a  soft-soil  model  and  particle  response  model.  The  software  apparent  dependence 
on  other  packages  kept  if  from  scoring  higher. 

Organization  E 
Threshold 

The  software  scored  Threshold  because  Geometry  can  be  imported  from  CAD  for  both 
vehicle  and  terrain.  Also  it  has  an  internal  integrated  EEA  solver  that  can  handle 
geometric  nonlinearities.  The  standard  terrain  definition  is  built  on  Bekker 
terramechanics,  but  a  DEM  approach  is  being  developed.  The  software  has  a  highly 
developed  track  modeling  system,  but  does  not  currently  have  an  off-road  tire  -  this 
feature  is  under  development. 
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Accuracy  -  Physics  Based 

Organization 

G 

Objective 

The  software  scored  Objective  because  it  can  import  CAD  models,  and  an  extension  of 
the  base  software  gives  pre-defined  2-D  and  3-D  contact  and  clearance  between 
arbitrary  bodies  like  parts  of  the  vehicle  as  well  as  between  vehicle  and  terrain.  The 
software  has  integrated  deformable  bodies,  ANCF  elements,  and  a  linear  modal  solver. 

It  can  be  linked  with  external  software  to  solve  material  and  geometric  non-linearities. 
The  software  does  not  natively  include  a  DEM  soil  model,  but  it  has  been  successfully 
integrated  with  other  software  for  high-fidelity  soft  soil  model  efforts.  It  offers  two  off¬ 
road  tire  models:  its  own  proprietary  tire  model  and  ETire. 

Organization 

H 

Below 

Threshold 

The  software  was  scored  Below  Threshold  because  it  has  realistic  graphics,  but  does  not 
appear  to  be  able  to  import  vehicle  geometry  or  parts  for  a  CAD  program  -  vehicles  are 
defined  in  text  XML  files.  The  environment  can  be  imported  from  a  GeoTIFF  file.  The 
software  allows  for  contact  between  the  vehicle  and  the  terrain  other  than  the  tires  or 
tracks,  but  the  objects  are  considered  rigid  bodies.  The  software  does  not  support 
deformable  bodies.  A  simple  deformable  off-road  tire  model  is  available  based  on  the 
Bekker-Wong  model,  but  detailed  tire  models  require  a  custom  software  plug-in. 

Organization  I 
Unacceptable 

The  software  scored  Unacceptable  because  it  uses  only  generalized  vehicle  models  - 
geometry  cannot  be  imported  from  CAD.  The  environment  can  be  imported  through 
multiple  formats,  however.  It  does  not  currently  support  deformable  bodies.  The 
software  has  a  multidisc  tire  model  that  determines  the  tire  deformation  from  the 
intersection  of  the  tire  with  the  polygons  that  define  the  terrain.  The  tire-soil  or  track- 
soil  interactions  have  been  modeled  using  Bekker’s  equation  and  shear  displacement. 
The  software  is  targeted  for  real-time  simulation  and  not  highly  detailed  FEA/DEM. 

Organization  J 
Objective 

The  software  was  scored  as  Objective  because  it  can  import  CAD  geometry  and  part 
interaction  can  be  rigid  or  flexible.  OpenCRG  is  used  to  import  the  environment,  and 
this  geometry  can  interact  with  vehicle  parts.  Simple,  flexible  elements  can  be  used  for 
quick  model  development  or  for  when  they  provide  sufficient  fidelity.  Modal  reduction 
of  flexible  components  and  non-linear  deformation  are  possible  with  external  software. 
The  software  includes  a  modified  Bekker  soft  soil  model.  A  high-fidelity  DEM  soil 
model  is  possible  through  co-simulation  with  external  software. 

Organization 

K 

Objective 

The  software  was  scored  Objective  because  it  can  import  vehicle  models  from  CAD 
programs  and  environmental  data  can  be  imported  and  converted  to  a  mesh.  The 
software  accounts  for  any  contact  between  a  vehicle  and  the  terrain.  It  also  has  flexible 
body  simulation  capabilities  using  the  ANCF  and  the  co-rotational  finite  element 
method.  Solvers  of  the  ANCF  and  co-rotational  non-linear  finite  elements  are  fully 
integrated.  The  software  has  a  simple  tire  model  but  is  being  extended  to  a  deformable 
tire  using  ANCF.  It  can  co-simulate  with  external  models  like  ETire.  It  has 
deformable/flowing  terrain  capabilities. 

Organization 

L 

Below 

Threshold 

The  software  scored  Below  Threshold  because  it  is  a  steady  state  2-D  model.  It  only 
has  2-D  vehicle  geometry  and  the  terrain  is  assumed  to  be  homogeneous  with  constant 
characteristics.  The  software  has  a  simple  flexible  tire  model  and  a  track  model 
described  through  longitudinal  stiffness,  but  cannot  interface  with  an  external  program 
for  detailed  analysis.  It  uses  Bekker-Wong  terramechanics;  the  terrain  is  assumed  to  be 
homogeneous  with  constant  characteristics. 

Validation  Through  Measurement  attributes  were  scored  according  to  the  software’s  ability  to  track  and 
correlate  simulation  results  with  recorded  test  results.  Both  vehicle  center  of  gravity  gross  motion  and 
individual  component  (e.g.,  wheel  /  damper  travel)  should  be  available. 
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Table  10-8.  Accuracy  -  Validation  Through  Measurement 


Accuracy  -  Validation  Throudi  Measurement 

Organization 

A 

Threshold 

The  software  scored  Threshold  because  it  supports  all  levels  of  detail  for  driveline 
modeling,  including  engine,  transmission  (manual  /  auto  /  CVT  /  etc.),  hybrid  electric 
drivelines,  torque  converters,  differentials,  and  transfer  cases.  All  parts  are  modeled 
with  physics  principles,  as  well  as  all-wheel  drive  dynamics  and  multi-axle  vehicles. 
The  software  can  handle  the  suspension  geometry,  but  the  spring/damper  model  isn’t 
thoroughly  discussed.  However  advanced  control  systems  require  3rd-party  software. 

Organization 

B 

Objective 

The  software  scored  Objective  because  it  allows  advanced  controls  through  JAVA  or 
Python  scripting  which  run  concurrently  with  the  simulation  and  can  read  the  system 
dynamic  response  (including  displacements,  deflections,  angles,  speeds,  forces,  etc.)  and 
generate  controller  actuator  forces.  HIE  is  supported.  It  has  detailed  powertrain 
modeling  (hybrids,  torque  converters,  transfer  cases,  diffs,  scripts  for  locking  the 
differentials,  all-wheel  drive,  and  clutches)  and  full  kinematic  engine  model.  It  also 
includes  various  suspension  systems  (double  wishbone,  McPherson  strut,  leaf-spring, 
walking  beam,  etc.).  The  software  models  suspension  deflection  and  vibrations. 

Organization 

C 

Threshold 

The  software  scored  Threshold  because  the  engine  and  other  drive  train  components, 
which  include  torque  converter,  transmission,  differentials,  transfer  cases  are  modeled. 
Electric  drive  is  available  but  full  hybrid  not.  Advanced  controls  can  be  created  C-H-  or 
Python,  or  implemented  in  Matlab/Simulink.  A  simulated  driver  is  included  based  on 
PID  controllers  for  speed,  steering,  etc.  The  suspension  can  be  modeled,  but  does  not 
appear  to  allow  flexible  joints  or  complex  designs. 

Organization 

D 

Below 

Threshold 

The  software  scored  Below  Threshold  because  it  is  capable  of  integration  with  an 
external  motor  controller  or  hardware.  It  contains  navigation  and  collision  detection 
algorithms  for  autonomous  vehicle  mobility  and  manipulation,  but  not  a  simulated 
driver.  Components  are  modeled  with  look-up  tables,  thus  the  simulation  lacks  detail. 
The  software  can  model  the  suspension,  but  it  requires  coding  to  run  efficiently. 

Organization 

E 

Above 

Threshold 

The  software  scored  Above  Threshold  because  open  and  closed  loop  control  is  possible 
and  implemented.  All  driveline  dynamics  are  modeled  with  a  scalable  level  of  detail, 
ranging  from  a  simple  throttle  with  first  order  dynamics  to  complete  air  path 
management  with  in-cylinder  representation  using  an  extension,  also  including  either 
rigid  connections  or  flexible  multibody  components  in  all  subsystems.  The  software 
includes  30  suspension  topologies.  Compliant  bushings  are  incorporated  and  active 
controls  are  possible,  however  they  require  verification. 

Organization  F 

Below 

Threshold 

The  software  scored  Below  Threshold  because  it  includes  a  module  to  model  advanced 
control  systems  that  is  similar  in  functionality  to  Simulink.  It  has  HIE  capability  with 
an  offered  extension.  It  does  not  have  preconfigured  templates  for  drive  trains  though, 
and  the  suspension  must  be  modeled  manually  by  the  user. 
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Accuracy  -  Validation  Throujiil  Measurement 

Organization 

G 

Above 

Threshold 

The  software  scored  Above  Threshold  because  position  /  forces  can  be  monitored  for  all 
components,  and  sensing  is  used  for  a  simulated  driver.  Advanced  controls  require  co¬ 
simulation  with  Matlab  or  similar  FMI  compliant  software.  Several  pre-defined 
transmission  types  are  available:  manual,  automatic  (with  torque  converter),  robotized 
manual,  hybrid,  and  simple  torque,  or  users  are  free  to  customize  transmission  models. 
Differentials,  transfer  cases,  AWD,  and  multi-axle  dynamics  can  be  explicitly  modeled 
in  as  much  detail  as  the  user  requires.  The  software  can  create  and  modify  fully 
parametric  templates  interactively  by  combining  low  level  primitives  (parts,  joints, 
forces)  and  higher  level  objects  (leaf-springs,  struts,  stabilizer-bars).  However  HIL 
requires  hard  coding. 

Organization 

H 

Below 

Threshold 

The  software  scored  Below  Threshold  because  advanced  controls  are  possible,  but  they 
require  a  plug-in.  A  simulated  driver  is  included  via  a  closed  loop  PID  system.  The 
software  can  model  unique  suspensions  but  it  appears  to  require  custom  code.  The  drive 
train  model  appears  to  be  limited  to  a  torque-speed-efficiency  look-up  table.  It  only  has 
rudimentary  HIL  capability. 

Organization  I 

Below 

Threshold 

The  software  scored  Below  Threshold  because  it  is  proficient  at  interfacing  with  user 
feedback  hardware  and  other  vehicle  hardware  can  be  integrated  in  a  similar  way.  It 
requires  Simulink  for  advanced  controls,  though.  The  software  is  focused  on  low- 
fidelity  models  for  real-time  simulation,  thus  the  driveline  systems  appear  to  be  look-up 
tables.  It  is  based  on  a  general  purpose  multibody  dynamics  code  that  can  be  used  to 
model  many  different  types  of  suspensions,  but  most  options  appear  to  require  hard 
coding  or  co-simulation. 

Organization  J 
Objective 

The  software  scored  Objective  because  it  has  a  simulated  driver  and  complete 
Driver/Software/Hardware-in-the-Loop  capabilities  with  a  program  extension  or 
through  interfacing  with  MATLAB.  The  software  provides  a  scalable  simulation 
environment,  allowing  optimization  between  fidelity  and  effort  in  simulation  time  or 
modeling  effort.  It  allows  creating  unique  suspension  designs.  Rigid  body  modes  of 
obstacles  are  taken  into  account  for  their  movement  on  collision.  With  contact 
modeling,  the  contact  forces  are  based  on  the  Hertz  theory.  Deformation  is  taken  into 
account  with  a  more  detailed  modal  or  FE  approach. 

Organization 

K 

Threshold 

The  software  scored  Threshold  because  it  is  capable  of  high-fidelity  modeling  of  drive 
train  and  suspension  components,  but  editing  text  files  and/or  custom  code  is  required. 
HIL,  SIL,  and  advanced  controls  have  been  implemented  but  require  either  co¬ 
simulation  or  custom  code. 

Organization 

L 

Below 

Threshold 

The  software  scored  Below  Threshold  because  it  can  model  pivot-arm  or  translational 
spring  suspensions,  with  linear  or  non-linear  load  deflection  characteristics.  It  does  not 
have  control  systems  or  a  driver  model,  and  thus  cannot  simulate  HIL/SIL/DIL  testing. 
Also  the  software  does  not  model  powertrain  subsystems,  look-up  tables  are  used. 
High-fidelity  modeling  is  not  possible. 

Supports  Time  and  Frequency  Domain  Analysis  attributes  were  scored  according  to  the  software’s  ability  to 
analyze  model  reaction  both  on-tbe-fly  and  in  post-processing.  Tbe  real-time  data  should  allow  tbe 
replication  of  complex  interactions  sucb  as  resonance  and  traction  bop.  Additional  post-processing  techniques 
should  be  available,  such  as  SRS,  PVSS,  durability  stress/strain  life  cycles,  etc. 
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Table  10-9.  Accuracy  -  Supports  Time  and  Frequency  Domain  Analysis 


Accuracy  -  Supports  Time  and  Frefluency  Domain  Analysis 

Organization 

A 

Objective 

The  software  scored  Objective  because  sensors  can  be  placed  anywhere  in  the  model  to 
extract  test  data,  before  or  after  the  simulation.  Results  can  be  graphically  displayed  in 
an  animation.  It  supports  order  analysis,  EET,  contribution  plots,  and  3-D  display  or 
results.  Optimization  can  be  done  through  co-simulation. 

Organization 

B 

Objective 

The  software  scored  Objective  because  it  is  able  to  create  animations,  plots,  and 
performing  various  data  analyses  including  averaging/smoothing  and  EET.  Data  can  be 
displayed  internally  or  exported  for  further  analysis.  It  is  also  capable  of  running 
Design  of  Experiment  (DOE),  stochastic  analysis,  and  parametric  studies  internally. 

Organization 

C 

Tbresbold 

The  software  scored  Threshold  because  time  domain  plots  and  animations  can  be 
created  natively.  It  includes  support  for  parametric  studies  of  the  model.  It  does  not 
directly  provide  frequency  domain  analysis,  but  the  test  data  can  be  exported  for 
complex  analysis. 

Organization 

D 

Objective 

The  software  scored  Objective  because  it  includes  time  domain  data  logging  and 
creation  of  movie  files.  Post  analysis  can  be  performed  using  Python  scientific 
computation  modules.  There  are  modules  for  Monte  Carlo  analysis  available  for 
parametric  sensitivity  and  uncertainty  analysis.  The  user  can  specify  the  range  and 
statistics  for  the  parameter  space  to  be  swept  through. 

Organization 

E 

Threshold 

The  software  was  Threshold  because  it  is  capable  of  plotting  and  post-processing, 
including  frequency  analysis,  but  it  requires  either  scripting  or  data  export.  Robust 
design  and  statistical  engineering  methods  are  integrated  in  the  software,  or  can  be 
achieved  through  co-simulation. 

Organization  E 
Threshold 

The  software  scored  Threshold  because  it  includes  extensive  model  parameterization 
and  DOE  capabilities.  It  also  includes  time-  and  frequency-domain  analysis  as  well  as 
animations.  There  was  a  lack  of  detail  in  their  response,  however,  so  specific 
capabilities  are  unclear. 

Organization 

G 

Objective 

The  software  was  scored  Objective  for  its  time-  and  frequency-domain  analysis.  It  is 
capable  of  simple  time  history  plots,  applying  sensors  to  any  point  in  the  model  to 
extract  forces  and  motion  through  the  simulation.  It  is  capable  of  EET  and  PSD 
analysis.  It  also  supports  DOE,  Monte  Carlo  analysis,  and  model  parameterization  and 
optimization  internally. 

Organization 

H 

Below 

Threshold 

The  software  was  scored  Below  Threshold  because  it  is  capable  of  creating  animations, 
but  does  not  have  complex  time-  or  frequency-domain  capabilities.  Simulations  can  be 
looped  to  vary  input  variables,  but  more  complex  DOE  is  still  under  development. 

Organization  I 

Below 

Threshold 

The  software  was  scored  Below  Threshold  because  it  only  has  low-level  time-domain 
analysis  and  no  frequency-domain  analysis.  There  appears  to  be  extensive  support  for 
animations,  including  overlaying  graphs  with  the  simulation.  There  are  no  internal 
methods  for  optimization  or  DOE,  but  it  is  possible  through  3rd-party  software. 

Organization  J 
Threshold 

The  software  was  scored  Threshold  because  it  has  extensive  post-processing 
capabilities,  including  a  dynamic  link  to  time-domain  curves  and  frequency-domain 
calculations.  No  examples  were  given,  however.  Methods  such  as  DOE  and  Monte 
Carlo  simulations  are  available  through  a  program  extension. 
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Accuracy  -  Supports  Time  and  Frequency  Domain  Analysis 

Organization 

K 

Threshold 

The  software  was  scored  Threshold  because  it  is  capable  of  time-and  frequency-domain 
plots,  but  only  through  custom  coding  or  linking  with  external  software.  It  is  capable  of 
creating  animations  via  two  integrated  methods,  as  well  as  displaying  data  with  the 
animation.  There  are  no  DOE  or  optimization  routines  built  into  the  software,  but  it  is 
possible  through  custom  code. 

Organization 

L 

Unacceptable 

The  software  scored  Unacceptable  because  it  is  purely  a  steady-state  model.  No  time- 
or  frequency-domain  analysis  is  possible.  It  also  does  not  currently  have  any  methods 
for  DOE,  parameterization,  or  optimization. 

10.5.2  Scoring.  Measures  of  Effectiveness:  Flexibility 

The  MOE  flexibility  had  three  measures  of  performance  that  were  scored  using  RFI  feedback  from  the 
vendors.  The  MOPs  reviewed  were  Template  Based,  Wheeled  /  Tracked  /  Amphibious  Vehicles,  and 
Automotive  Subsystems. 

Template  Based  attributes  were  scored  according  to  the  usability  of  the  software.  The  software  must  allow 
the  building  of  a  vehicle  from  components,  subsystems,  and  systems  that  are  available  in  a  template  database 
included  with  the  software.  Different  components,  subsystems,  and  systems  should  be  able  to  be  swapped  in 
order  to  evaluate  the  change  of  performance.  The  process  of  building  the  vehicle  model  should  be  done  in  a 
graphical  user  interface  environment.  While  custom  coding  may  be  available  for  advanced  users,  novice  users 
should  be  able  to  construct  a  representative  vehicle  using  the  GUI. 


Table  10-10.  Flexibility  -  Template  Based 


Flexibility  -  Template  Based 

Organization  A 
Objective 

The  software  was  scored  Objective  because  it  has  a  customizable  sub-mechanism 
structure  included  with  its  vehicle  database.  This  includes  connecting  multiple  levels 
of  sub-mechanisms.  Editing  of  the  sub-mechanisms  is  possible  from  the  main  model. 

It  supports  graphical  and  text  based  editing  of  the  model,  including  editing  the  3-D 
geometry. 

Organization  B 
Above 

Threshold 

The  software  was  scored  Above  Threshold  because  it  has  high  potential  for  individual 
components,  but  there  isn't  an  extensive  library  of  components  ready  for  the  template 
because  the  market  penetration  appears  to  be  small.  Its  GUI  includes  a 

template/wizard/spreadsheet  editor  that  uses  figures  and  tables  to  show  graphically  the 
geometric  parameters  of  the  sub-model. 

Organization  C 
Threshold 

The  software  was  scored  Threshold  because  it  has  a  thorough  list  of  major  vehicle 
systems,  but  does  not  model  individual  components.  It  provides  access  to  aspects  of 
the  simulation  through  a  point-and-click  graphical  user  interface.  In  addition,  it  also 
includes  live  test  and  validation  capabilities  to  edit  mechanisms  while  running  the 
simulation  to  see  behavior  and  changes  immediately,  without  having  to  run  an  external 
application. 

Organization  D 
Below 

Threshold 

The  software  was  scored  as  Below  Threshold  because  any  number/level  of  systems  and 
components  can  be  modeled,  but  the  primary  method  is  through  scripting.  There  is  a 
“tree-augmented”  approach  to  creating  the  model  which  appears  to  be  graphically 
implemented,  but  the  resulting  model  sacrifices  execution  speed  at  run  time. 

Organization  E 
Objective 

The  software  was  scored  Objective  because  systems,  subsystems,  and  components  are 
available  via  templates  and  libraries.  The  GUI  allows  building  models  of  different 
fidelities,  adapting  the  modeling  process  to  advanced  and  novice  users. 
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Flexibility  -  Template  Based 

Organization  F 
Threshold 

The  software  was  scored  Threshold  because  it  allows  for  highly  detailed  modeling  of 
the  vehicle  track  and  suspension,  but  information  is  limited  regarding  other  systems. 
Properties  of  the  track  related  components  can  be  defined  through  a  wizard  type 
interface. 

Organization  G 
Objective 

The  software  was  scored  Objective  because  it  is  designed  for  template -based  modeling 
of  system,  subsystem,  and  component  level  interactions.  The  template  builder 
environment  features  a  guided  user  interface,  symmetry  support,  and  an  interactive 
graphical  model  view. 

Organization  H 
Unacceptable 

The  software  was  scored  Unacceptable  because  component  modeling  is  not  possible, 
and  system  /  sub-system  models  are  typically  look-up  tables.  There  is  no  GUI,  the 
model  is  created  completely  through  text  files. 

Organization  I 
Unacceptable 

The  software  was  scored  Unacceptable  because  while  the  vehicle  can  be  split  into 
systems  and  sub-systems,  the  models  are  low-fidelity  look-up  tables.  If  high-fidelity 
component  modeling  is  needed  then  co-simulation  is  required.  A  program  extension  is 
available  to  graphically  build  a  model  of  a  wheeled  vehicle  (apparently  not  available 
for  tracks),  but  the  results  have  not  been  verified. 

Organization  J 
Objective 

The  software  scored  Objective  because  it  includes  system,  subsystem,  and  components 
that  can  be  swapped  for  various  levels  of  fidelity.  The  vehicle  systems,  terrain  data, 
and  mission  profiles  can  all  be  edited  with  GUI  based  “vertical”  applications. 

Organization  K 
Threshold 

The  software  was  scored  Threshold  because  full  systems,  subsystem,  and  component 
levels  are  available  in  templates.  The  templates  are  in  the  form  of  text  files,  however. 
There  is  no  GUI  for  the  software. 

Organization  L 
Unacceptable 

The  software  was  scored  Unacceptable  because  while  it  does  have  a  GUI,  only  basic 
vehicle  and  environmental  parameters  are  used.  The  powertrain  is  modeled  as  look-up 
table,  subsystem  and  component  level  modeling  is  not  possible. 

Wheeled  /  Tracked  /  Amphibious  vehicle  attributes  were  scored  according  to  the  software’s  ability  to  model 
numerous  types  of  vehicles  in  the  diverse  environment  required.  The  type  of  the  tire  model  was  a  factor  (on¬ 
road  vs  off -road),  as  well  as  the  detail  used.  Likewise  the  ability  to  model  different  designs  of  tracks  (single 
pin,  double  pin,  “live,”  “dead,”  rubber  band,  etc.)  is  required.  The  software  must  be  able  to  simulate  operation 
in  land,  sea,  and  the  littoral  transition. 


Table  10-11.  Flexibility  -  Wheeled  or  Tracked  Vehicles 


Flexibility  -  Wheeled  or  Tracked  Vehicles 

Organization  A 
Unacceptable 

The  software  was  scored  Unacceptable  because  while  it  includes  multiple  “standard” 
tire  models  used  for  paved  scenarios,  there  is  no  off-road  tire  model.  Tracked 
vehicles  are  not  supported  at  all.  Hydrodynamic  modeling  is  possible,  but  doesn’t 
appear  to  be  validated. 

Organization  B 
Objective 

The  software  scored  Objective  because  there  are  two  off-road  tire  models  available. 
Multiple  designs  of  tracks  are  also  available.  Smoothed  Particle  Hydrodynamics 
(SPH)  is  included  for  modeling  fluid  interaction  with  rigid  and  flexible  bodies. 
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Flexibility  -  Wheeled  or  Tracked  Vehicles 

Organization  C 
Threshold 

The  software  was  scored  Threshold  because  while  an  off-road  tire  model  is  available, 
it  is  low-fidelity.  Similar  to  wheels,  track  models  are  available  but  they  do  not 
differentiate  between  single-  and  double-pin  designs.  Both  wheels  and  tracks  could 
be  extended  through  custom  coding,  however.  The  software  is  more  detailed  with  its 
hydrodynamic  model,  though,  incorporating  drag,  lift,  buoyancy,  and  transition  to 
land. 

Organization  D 
Unacceptable 

The  software  was  scored  Unacceptable  because  only  the  Eiala  tire  model  is  included. 
The  software  does  not  include  any  track  models.  Likewise  only  limited  aquatic 
modeling  has  been  done,  with  no  experience  for  the  sea-to-shore  transition.  Custom 
code  could  be  used  as  a  plug-in  for  all  three  criteria,  however. 

Organization  E 
Below  Threshold 

The  software  was  scored  Below  Threshold  because  while  it  does  not  have  a  native 
off-road  tire  model,  it  has  interfaces  with  standard  Delft  and  ETire  models.  There  are 
not  pre-designed  tracks,  but  accurate  models  could  be  built  from  parts  or  imported 
from  CAD  designs.  Hydrodynamic  forces  have  been  done,  but  are  not  included  as 
part  of  the  library. 

Organization  E 
Threshold 

The  software  was  scored  Threshold  because  it  includes  several  high-fidelity  track 
models.  There  is  currently  only  a  low-fidelity  tire  model,  however.  Hydrodynamic 
forces  are  modeled  by  co-simulating  with  a  third  party  software  using  smoothed 
particle  hydrodynamics.  More  development  would  be  needed  for  the  transition 
phase,  however. 

Organization  G 
Above  Threshold 

The  software  was  scored  Above  Threshold  because  low  and  high-fidelity  tire  models 
are  included.  Various  templates  are  available  for  tracks  and  track  suspensions  are 
included.  Water-based  effects  are  only  basic,  however.  Explicit  forces  can  be 
defined,  or  higher  fidelity  achieved  through  co-simulation. 

Organization  H 
Unacceptable 

The  software  was  scored  Unacceptable  because  only  low-fidelity  models  are 
available  for  both  tires  and  tracks.  Hydrodynamics  are  not  offered. 

Organization  I 
Unacceptable 

The  software  was  scored  Unacceptable  because  it  is  only  capable  of  a  super-element 
track  model.  It  has  a  multi-disc  tire  model,  but  it  doesn’t  support  high-fidelity 
analysis.  It  has  not  been  used  with  any  hydrodynamic  forces. 

Organization  J 
Objective 

The  software  was  scored  Objective  because  a  dedicated  off-road  tire  model  was 
developed  and  validated.  It  includes  both  low-  and  high-fidelity  methods  to  create 
custom  tracks.  Hydrodynamic  forces  for  buoyancy  and  drag  have  been  modeled. 

Organization  K 
Above  Threshold 

The  software  was  scored  Above  Threshold  because  high-fidelity  modeling  of  tracked 
vehicles  is  possible  through  text  templates  of  the  track  suspension  and  components. 
Only  a  simplified  off-road  tire  is  currently  available;  a  high-fidelity  tire  is  being 
developed  as  a  deformable  body.  Hydrodynamic  forces  are  evaluated  using  a 
Lagrangian  fluid  formulation  similar  to  Smoothed  Particle  Hydrodynamics  (SPH). 

Organization  L 
Unacceptable 

The  software  was  scored  Unacceptable  because  it  only  includes  basic  models  for  both 
tires  and  tracks.  The  software  is  not  designed  for  predicting  the  fording  or 
amphibious  performance  of  off-road  vehicles. 

Automotive  subsystems  attributes  were  scored  according  to  the  software’s  ability  to  accurately  create  a  model 
down  to  tbe  component  level.  Interactions  between  components  are  considered.  Linear  and  non-linear 
characteristics  should  be  possible.  Control  systems  may  be  required  for  active  suspension,  braking,  stability, 
and  traction  control  systems.  Vehicle  and  environmental  feedback  will  be  used  for  autonomous  vehicle 
simulation  and  control.  Hardware  and  software  in  the  loop  may  be  required. 
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Table  10-12.  Flexibility  -  Automotive  Subsystems 


Flexibility  -  Automotive  Subsystems 


Organization  A 
Threshold 

The  software  was  scored  Threshold  because  it  supports  all  types  of  powertrains 
(gasoline  /  diesel  /  hybrid,  and  manual  /  auto  /  IVT).  Important  systems  such  as 
clutches,  torque  converters,  and  differentials  are  also  modeled  using  physics 
principles.  Control  systems  are  possible  through  internal  methods,  but  co-simulation 
may  be  more  effective  (it  is  EMI  compliant).  The  organization  did  not  respond  to  the 
Autonomous  Vehicles  questionnaire,  thus  it  cannot  be  evaluated  for  full  autonomous 
vehicle  development. 

Organization  B 
Threshold 

The  software  was  scored  Threshold  because  it  has  perhaps  the  most  detail  with  the 
engine  models,  going  down  to  moving  parts  and  inertias.  Active  controls,  various 
stability  systems,  etc.  are  implemented  via  JAVA  and  Python  scripting.  It  is  not  EMI 
compliant,  but  the  organization  was  open  to  developing  this  capability  if  needed. 
The  simulation  environment  has  the  detail  and  data  capturing  capabilities  needed  for 
autonomous  vehicle  operation,  but  this  hasn’t  been  done  yet  -  more  development 
may  be  needed. 

Organization  C 
Below  Threshold 

The  software  was  scored  Below  Threshold  because  it  includes  a  full  featured  graphics 
engine  with  a  detailed  environmental  condition  modeling,  including  time-of-day 
specifications,  shadow  casting,  cloud  cover,  night  time,  fog,  and  dust  particle 
modeling.  The  various  powertrain  configurations  are  all  possible,  but  in  low-fidelity 
look-up  table  form.  Controllers  are  possible,  but  require  custom  plug-ins  written  in 
C-H-  or  Python,  or  co-simulation  with  MATLAB/Simulink.  It  is  unclear  whether  the 
software  is  EMI  compliant. 

Organization  D 
Below  Threshold 

The  software  was  scored  Below  Threshold  because  the  powertrain  model  is  limited  to 
low-fidelity  look-up  tables.  Vehicle  controls  can  be  implemented  through  its 
application  program  interface  (API)  or  co-simulation  with  Simulink.  It  is  unclear 
whether  it  is  EMI  compliant,  however.  The  software  includes  high-fidelity  models 
for  mono  and  stereo  cameras,  but  does  not  seem  to  have  other  “sensors”. 

Organization  E 
Threshold 

The  software  was  scored  Threshold  because  it  includes  numerous  libraries  which 
allow  efficient  modeling  of  various  physical  systems:  vehicle  dynamics,  powertrain, 
electronics,  heating/cooling,  hydraulics,  pneumatics,  batteries,  and  specific  military 
ground  vehicle  libraries.  It  allows  four  methods  to  model  vehicle  system  controls  and 
communication,  ranging  from  importing  the  control  model  to  exporting  the  dynamics 
model  or  co-simulation.  The  software  is  EMI  compliant.  The  organization  did  not 
respond  to  the  Autonomous  Vehicles  questionnaire,  however. 

Organization  E 
Below  Threshold 

The  software  was  scored  Below  Threshold  because  while  it  is  capable  of  detailed 
powertrain  modeling,  it  does  not  include  templates  for  specific  systems  like  diesel  or 
hybrid  designs.  It  does  include  an  integrated  control  design  module,  or  is  capable  of 
co-simulation  with  Simulink.  It  isn’t  known  whether  the  software  is  FMI  compliant. 
Also  they  did  not  respond  to  the  Autonomous  Vehicle  questionnaire. 

Organization  G 
Above  Threshold 

The  software  was  scored  Above  Threshold  because  while  engine  dynamics  are 
typically  limited,  they  could  be  modeled.  A  full  set  of  templates  is  available  for 
different  transmission  types  with  subsystems  like  torque  converters  and  clutches.  The 
software  has  an  extension  for  designing  and  tuning  control  systems,  as  well  as  being 
FMI  compliant  and  able  to  link  to  Simulink.  The  software  is  not  designed  for 
simulating  autonomous  vehicles  interacting  with  the  environment,  but  it  could  be 
possible  though  co-simulation. 
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Flexibility  -  Automotive  Subsystems 

Organization  H 
Unacceptable 

The  software  was  scored  as  Unacceptable  because  the  engine  and  powertrain  are 
simply  modeled  with  look-up  tables.  Any  detailed  components  such  as  dampers  are 
controlled  either  with  basic  PID  controls  or  custom  code.  Its  vehicle  model  and 
environment  have  been  designed  around  real-time  simulation  of  autonomous 
vehicles.  The  software  is  not  FMI  compliant,  though. 

Organization  I 
Below  Threshold 

The  software  was  scored  as  Below  Threshold  because  while  it  does  offer  a  variety  of 
powertrain  options,  they  are  generalized,  low-fidelity  components  designed  for  fast 
simulation.  There  is  a  reference  to  linking  third  party  software,  but  it  is  not  explicitly 
stated  whether  the  software  is  FMI  compliant.  Control  systems  can  be  designed 
internally.  It  does  support  design  of  autonomous  systems,  however. 

Organization  J 
Above  Threshold 

The  software  was  scored  Above  Threshold  because  it  integrates  detailed  engine, 
powertrain,  and  system  controls  internally.  It  is  FMI  compliant,  and  can  link  with 
Simulink  if  desired.  The  software  has  been  designed  for  high-fidelity  mechanical 
simulation  rather  than  complex  environmental  interaction.  It  could  facilitate 
autonomous  vehicle  development,  but  would  require  co-simulation  with  external 
software. 

Organization  K 
Below  Threshold 

The  software  was  scored  Below  Threshold  because  while  it  is  capable  of  detailed 
modeling  of  any  system,  it  is  dependent  on  C-H-  coding.  This  applies  to  the  system 
control  as  well.  The  software  is  FMI  compliant,  so  the  controls  could  be  created  in 
external  software.  When  paired  with  a  related  graphics  package  it  is  capable  of 
autonomous  vehicle  development,  again  with  C-t-i-  coding. 

Organization  L 
Unacceptable 

The  software  was  scored  as  Unacceptable  because  the  engine  and  powertrain  are 
completely  generalized  as  look-up  tables.  As  a  steady-state  model  there  are  no 
system  control  systems  possible.  It  is  not  FMI  compliant.  Also  they  did  not  respond 
to  the  Autonomous  Vehicle  questionnaire,  but  given  the  nature  of  the  model  it  is  not 
suitable  for  developing  autonomous  systems. 

10.5.3  Scoring.  Measures  of  Effectiveness:  Cost 

The  MOE  cost  had  three  measures  of  performance  that  were  scored  using  RFI  feedback  from  the  vendors. 
The  MOPs  reviewed  were  License,  Run  Time,  and  Training. 

License  attributes  were  the  initial  cost  of  the  license  itself  and  any  additional  costs  that  would  be  incurred 
such  as  extra  software  toolboxes  that  would  be  needed  and  not  included  in  the  initial  offering  from  the 
organizations.  Other  attributes  were  the  level  of  support  that  would  be  included  in  the  initial  price,  scores  were 
decreased  if  support  was  not  included  in  the  initial  cost.  Score  reductions  were  also  given  to  organizations  that 
did  not  provide  support  at  all.  While  open  source  code  was  a  desired  attribute,  the  associated  software  support 
was  also  examined;  items  such  as  data  security  and  how  it  is  protected  were  reviewed.  Some  vendors  that 
offered  “free”  software  did  not  account  for  the  network  IT  personnel  and  time  that  would  be  required  by  the 
customer  to  accommodate  the  security  threats  when  this  function  was  built  in  to  other  more  expensive 
software  packages.  The  total  cost  was  calculated  over  a  5  year  period  and  the  cost  to  own  per  year  was  then 
scored.  Additional  metric  was  each  organization’s  ability  to  provide  a  NATO  trial  license  for  evaluation 
purposes  of  their  software. 
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TablelO-13.  Cost,  Maintenance  and  Training  -  Licensing/5  Yr.  Cost 


Cost,  Maintenance  and  Training  -  Licensing/5  Yr.  Cost 

Organization 

A 

Below 

Threshold 

Organization  A  receives  a  score  of  Below  Threshold.  While  they  have  the  ability  to 
provide  NATO  with  a  license  for  evaluation  purposes  at  no  cost,  the  cost  per  year  over  5 
years  is  below  threshold.  Additional  costs  are  included  each  year  for  separate  software 
licenses  that  are  required  to  supplement  the  software  operations. 

Organization 

B 

Objective 

Organization  B  meets  Objective  with  the  ability  to  provide  NATO  with  a  license  for 
evaluation  purposes  at  no  cost,  and  showing  a  cost  per  year  over  a  five  year  period  that 
meets  objective.  Additional  costs  are  included  each  year  for  separate  software  licenses 
that  are  required  to  supplement  the  software  operations. 

Organization 

C 

Above 

Threshold 

Organization  C  is  Above  Threshold  illustrating  the  ability  to  provide  NATO  with  a 
license  for  evaluation  purposes  at  no  cost.  Yearly  cost  meets  objective  cost  per  year  over 
a  five  year  period.  The  software  does  not  require  additional  software  licenses  to  support 
operations. 

Organization 

D 

Unacceptable 

Organization  D  receives  a  score  of  Unacceptable  as  they  did  not  communicate  the 
ability  to  provide  a  trial  license  at  no  cost  for  evaluation  of  their  software.  Organization 
D  did  provide  a  threshold  cost  per  year  over  5  years.  Additional  costs  are  included  each 
year  for  separate  software  licenses  which  will  be  required  to  supplement  the  software 
operations. 

Organization 

E 

Unacceptable 

Organization  E  receives  a  score  of  Unacceptable  as  they  did  not  communicate  the  ability 
to  provide  a  trial  license  at  no  cost  for  evaluation  of  their  software.  Organization  E  does 
meet  objective  cost  per  year  over  five  years.  Additional  costs  are  included  each  year  for 
separate  software  licenses  which  will  be  required  to  supplement  the  software  operations. 

Organization  E 
Unacceptable 

Organization  E  receives  a  score  of  Unacceptable  as  they  did  not  communicate  the  ability 
to  provide  a  trial  license  at  no  cost  for  evaluation  of  their  software.  Organization  E  did 
provide  an  above  threshold  cost  per  year  over  5  years.  The  software  does  not  require 
additional  software  licenses  to  operate  as  desired. 

Organization 

G 

Above 

Threshold 

Organization  G  scores  Above  Threshold  illustrating  the  ability  to  provide  NATO  with  a 
license  for  evaluation  purposes  at  no  cost.  Organization  G  meets  threshold  cost  per  year 
over  a  five  year  period.  The  software  does  not  require  additional  licenses  to  operate  as 
desired. 

Organization 

H 

Above 

Threshold 

Organization  H  scores  Above  Threshold  overall.  The  price  meets  objective  cost  per  year 
over  five  years.  Additional  costs  are  included  each  year  for  separate  software  licenses 
which  are  required  to  supplement  the  software  operations.  Organization  H  is  willing  to 
provide  a  six  month  license  at  no  cost  for  purposes  of  evaluating  the  software. 

Organization  I 
Objective 

Organization  I  meets  Objective  with  the  ability  to  provide  NATO  with  a  license  for 
evaluation  purposes  at  no  cost.  Organization  I  also  meets  objective  cost  per  year  over  a 
five  year  period.  The  software  does  not  require  additional  software  licenses  to  operate 
as  desired. 

Organization  J 

Above 

Threshold 

Organization  J  is  Above  Threshold  illustrating  the  ability  to  provide  NATO  with  a 
license  for  evaluation  purposes  at  no  cost.  Organization  J  meets  threshold  cost  per  year 
over  a  five  year  period.  The  software  does  not  require  additional  software  licenses  to 
operate  as  desired. 

Organization 

K 

Above 

Threshold 

Organization  K  is  Above  Threshold  illustrating  the  ability  to  provide  NATO  with  a 
license  for  evaluation  purposes  at  no  cost.  Yearly  cost  is  above  threshold  per  year  over  a 
five  year  period.  Additional  costs  are  included  each  year  for  separate  software  licenses 
which  are  required  to  supplement  the  software  operations. 
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Cost,  Maintenance  and  Training  -  Licensing/5  Yr.  Cost 

Organization 

L 

Unacceptable 

Organization  L  scores  an  unacceptable  due  to  not  have  a  pricing  structure  and  indicating 
that  number  of  licenses  would  dictate  tbe  cost  which  would  have  to  be  negotiated.  No 
information  was  given  with  regard  to  trial  licenses  and  their  associated  cost. 

Run  time  attributes  were  scored  on  the  software’s  ability  to  support  multi-core/multi -processor,  shared 
memory  through  parallel  computers/nodes.  This  ability  is  twofold  with  regard  to  customer  costs.  First  the 
multi-core/multi-processor  approach  allows  parallel  computers  working  together  to  decrease  simulation  time. 
Second  the  use  of  high  powered  computers  is  not  a  necessity  when  using  this  type  of  processing  therefore 
decreasing  operational  costs  for  the  customer.  Finally,  each  of  the  vendor’s  offerings  was  examined  to  see 
their  compatibility  with  Linux  and  Windows  based  operating  systems.  Compatibility  with  both  was  scored 
bigber. 


Table  10-14,  Cost,  Maintenance  and  Training  -  Run  Time 


Cost,  Maintenance  and  Training  -  Run  Time 

Organization 

A 

Above 

Threshold 

Organization  A  is  Above  Threshold  with  the  ability  to  operate  on  Windows  and  Linux 
operating  systems.  The  software  supports  multi-core  operations  up  to  a  16  core 
maximum  for  efficiency  purposes. 

Organization 

B 

Threshold 

Organization  B  meets  Threshold  because  the  solver  currently  runs  on  shared-memory 
parallel  computers/nodes,  including  multi-core/multiprocessor  computers  and  Intel  Phi 
coprocessors.  The  solver  runs  on  both  Windows  and  Linux.  However,  the  pre-  and  post¬ 
processor  runs  only  on  Windows. 

Organization 

C 

Above 

Threshold 

Organization  C  is  Above  Threshold  with  the  ability  to  operate  on  Windows  and  Linux 
operating  systems.  The  software  supports  multi-core  operations  up  to  a  16  core 
maximum  for  efficiency  purposes.  Organization  C  states  that  collision  detection  and 
multiple  vehicles  or  multiple  experiments  can  always  be  solved  in  parallel.  The  amount 
of  parallelism,  however,  depends  on  the  dynamics  system  being  solved. 

Organization 

D 

Below 

Threshold 

Organization  D  is  Below  Threshold  since  its  solver  is  primarily  targeted  for  workstation 
and  embedded  use  -  not  parallel  processing.  The  software  is  only  compatible  with  Linux 
operating  systems. 

Organization 

E 

Above 

Threshold 

Organization  E  is  Above  Threshold  because  its  software  can  perform  parallel 
computations  utilizing  numerous  cores  and  is  compatible  with  both  Windows  and  Linux 
operating  systems. 

Organization  E 

Above 

Threshold 

Organization  E  is  Above  Threshold  because  its  software  can  perform  parallel 
computations  utilizing  numerous  cores  and  is  compatible  with  both  Windows  and  Linux 
operating  systems,  however  the  graphical  user  interface  is  only  supported  on  Windows 
at  this  time. 

Organization 

G 

Objective 

Organization  G  met  objective  because  its  software  supports  multi -core  parallel 
computations.  Utilizes  64-bit  operating  platforms  to  increase  performance.  Compatible 
with  both  Windows  and  Linux  operating  systems. 

Organization 

H 

Below 

Threshold 

Organization  H  is  Below  Threshold.  Support  for  parallel  processing  does  not  exist  but  is 
under  development.  Linux  compatibility  also  under  development.  System  is  currently 
compatible  with  Windows  operating  system. 
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Cost,  Maintenance  and  Training  -  Run  Time 

Organization  I 

Below 

Threshold 

Organization  I  is  Below  Threshold  because  the  software  has  limited  support  for  multi¬ 
core  processes.  It  is  however,  compatible  with  both  Windows  and  Linux  operating 
systems. 

Organization  J 
Objective 

Organization  J  meets  Objective  because  the  software  can  run  in  parallel  with  multi-cores 
and  is  compatible  with  Windows  and  Linux  operating  systems. 

Organization 

K 

Above 

Threshold 

Organization  K  is  Above  Threshold  because  the  software  is  capable  of  parallel  multi¬ 
core  CPU  computing.  Organization  K’s  software  is  compatible  with  both  Windows  and 
Linux  operating  systems. 

Organization 

L 

Unacceptable 

Organization  L  receives  an  Unacceptable  score  because  the  software  does  not  perform 
parallel  processing  and  is  compatible  with  Windows  operating  system  only. 

Training  attributes  were  scored  on  the  vendor’s  level  of  support  and  how  that  support  was  structured. 
Questions  such  as,  did  the  vendor  have  sufficient  staff  to  be  able  to  travel  to  the  customers  site  for  training 
sessions,  did  the  vendor  have  the  staff  to  provide  support  via  telephone  or  videoconference,  did  the  vendor 
display  sufficient  market  penetration  to  exhibit  a  large  user  community  for  support.  Other  support  parameters 
were  examined,  such  as  the  amount  of  web-based  support  in  the  form  of  chat  rooms,  tutorials,  user  manuals 
etc. 


Table  10-15.  Cost,  Maintenance  and  Training  -  Training 


Cost,  Maintenance  and  Training  -  Training 

Organization 

A 

Threshold 

Organization  A  meets  Threshold  because  they  provide  automated  support  such  as  a 
website  with  Q&A  support  community,  message  boards  etc.  They  also  provide  support 
via  email,  WebEx,  and  phone.  Organization  A  was  not  forthcoming  on  whether  or  not 
they  physically  will  travel  to  a  site  and  provide  training. 

Organization 

B 

Below 

Threshold 

Organization  B  scores  Below  Threshold,  but  it  is  very  strong  in  technical  training  and 
support  through  internet  based  video  conferencing.  Organization  B  does  not  address  on 
site  support,  either  at  their  facility  or  the  customers.  Website  solutions  are  limited. 

Organization 

C 

Objective 

Organization  C  meets  Objective  and  will  provide  training  on  site  at  their  facility  or  the 
customers.  They  also  have  a  large  web  based  automated  training  capability  as  well  as 
live  support  via  email,  phone  and  video  conferencing. 

Organization 

D 

Below 

Threshold 

Organization  D  is  Below  Threshold.  Organization  D  will  host  visitors  at  its  site  to 
collaborate  on  joint  efforts  but  no  formal  training  is  offered. 

Organization 

E 

Unacceptable 

Organization  E  receives  an  Unacceptable  score  because  available  personnel  does  not 
constitute  a  large  mobile  training  force.  If  needed  Organization  E  can  “ramp  up”  efforts 
to  meet  tbe  needs  of  the  customer.  There  is  a  web-based  education  center  but  no 
interactive  support  is  offered. 

Organization  F 
Threshold 

Organization  F  meets  Threshold  by  offering  training  from  its  regional  offices,  either  on 
site  at  their  office  or  the  customers.  Automated  support  is  not  mentioned  but  interactive 
support  such  as  phone,  email,  video  conference  is  offered  but  for  additional  costs. 
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Organization 

G 

Objective 

Organization  G  meets  Objective  offering  on-site  training  at  their  site  or  the  customers. 
Organization  G  also  offers  an  online  “Knowledge  base”  for  Q&A,  blogs,  message 
centers,  etc.  Technical  support  is  also  offered  through  phone,  emails,  and  video 
conferencing. 

Organization 

H 

Unacceptable 

Organization  H  receives  an  Unacceptable.  Organization  H  will  not  provide  training  at 
their  site  or  the  customers.  There  is  a  website  that  can  be  utilized  to  contact  them  for  any 
issues  but  no  formal  training  is  mentioned. 

Organization  I 

Above 

Threshold 

Organization  I  is  Above  Threshold  because  it  will  provide  training  at  their  site  or  the 
customers.  They  offer  unlimited  emails  and  phone  support.  Automated  support  is 
limited. 

Organization  J 
Threshold 

Organization  J  meets  Threshold  because  they  provide  support  via  phone  or  email.  Web 
page  support  is  limited  and  Organization  J  does  not  support  onsite  training,  either  at 
their  facility  or  the  customers. 

Organization 

K 

Unacceptable 

Organization  K  receives  an  Unacceptable  score  because  it  utilizes  a  web  based  “issue 
tracker”  to  solve  problems  if  encountered.  No  live  support  via  phone  or  video 
conference  is  offered.  Organization  K  does  not  travel  to  customer  sites  and  does  not  host 
training. 

Organization 

L 

Above 

Threshold 

Organization  L  is  Above  Threshold  because  it  will  provide  training  on  site  at  either  their 
facility  or  the  customers.  Organization  L  also  provides  training  through  phone  calls, 
emails  and  video  conferencing.  Automated  support  such  as  web  pages  etc  is  limited. 

10.5.4  Scoring.  Measure  of  Effectiveness:  NATO  Specific  Applications 

The  MOE  NATO  Specific  Applications  MOE  was  supported  by  MOPs  that  looked  at  the  vendors  offering 
ability  to  support  unique  terrain  or  mission  definition,  the  availability  worldwide  of  the  vendors’  offerings  and 
what  world  wide  support  is  available  for  each. 

The  vendors  support  of  unique  terrain  or  mission  definitions  was  scored  based  on  the  software’s  ability  to 
provide  variable  terrain  in  a  three  dimensional  setting  with  options  to  customize  the  terrain  in  general  as  well 
as  provide  the  soil  properties  and  interaction  with  heterogeneous  soil  conditions.  Finally,  the  terrain  could 
further  be  altered  via  simulated  climate  conditions.  Two-dimensional  terrain  that  is  not  variable  was  deemed 
unacceptable. 


Table  10-16.  NATO  Specific  Applications  -  Supports  unique  terrain  or  mission  definition 


NATO  Specific  Applications  -  Supports  unique  terrain  or  mission  definition 

Organization 

A 

Threshold 

Organization  A  meets  Threshold  because  it  can  support  3-D  terrain  but  is  vague  on 
importing  GIS  type  data.  The  soil  data  is  UDF  and  would  need  some  additional 
conversion  to  implement. 

Organization 

B 

Objective 

Organization  B  meets  Objective.  It  can  support  3-D  terrain  and  imports  GIS  and 
converts  to  polygonal  surface  or  x,  y,  z  point  data  for  any  application.  The  software 
implements  DEM  and  can  be  used  to  specify  soil  conditions. 

Organization 

C 

Threshold 

Organization  C  meets  Threshold  because  it  can  support  3-D  terrain  but  would  take  some 
additional  development  for  GIS  and  NRMM  functionality.  Organization  C  does  provide 
sun/solar  variables  in  the  input  but  does  not  elaborate. 

UNCLASSIFIED:  Distribution  Statement  A.  Approved  for  public  release;  distribution  is  unlimited 


106 


NATO  Specific  Applications  -  Supports  unique  terrain  or  mission  definition 

Organization 

D 

Below 

Threshold 

Organization  D  is  Below  Threshold.  Organization  D  Supports  3-D  terrain  but  provides 
no  way  to  import  GIS  data.  Climate  and  soil  properties  are  not  included  in  the  base 
offering.  It  is  unclear  if  an  additional  module  is  available  that  supports  soil  properties. 

Organization 

E 

Unacceptable 

Organization  E  receives  an  Unacceptable  score  with  3-D  terrain  partially  supported 
using  3'“*  party  software.  Once  the  terrain  is  defined,  no  soft  soil  model  capability  exists 
so  it  is  not  possible  to  have  climatic  influences.  GIS  currently  cannot  be  imported. 

Organization  F 
Threshold 

Organization  F  meets  Threshold  because  3-D  terrain  is  supported  in  dimensions  only 
and  not  deformable.  GIS  data  can  be  imported;  soil  types  are  existent  but  need 
additional  development. 

Organization 

G 

Objective 

Organization  G  meets  Objective  with  numerous  3-D  formats  supported,  software  can 
import  all  GIS  data  using  3rd-party  software.  Contains  a  working  DEM  for  soil  property 
manipulations. 

Organization 

H 

Above 

Threshold 

Organization  H  is  Above  Threshold  because  3-D  terrain  and  import  of  GIS  data  is  fully 
supported.  Soil  is  specified  in  layers  and  the  response  was  unclear  as  to  how  the  soil 
properties  are  handled  for  each. 

Organization  I 

Above 

Threshold 

Organization  I  is  Above  Threshold  because  3-D  terrain  is  supported,  numerous  options 
for  importing  and  manipulating  GIS  type  data.  Allows  terrain  soil  type  definitions  with 
lookup  tables.  Soil  changes  with  climate  is  under  development. 

Organization  J 

Above 

Threshold 

Organization  J  is  Above  Threshold  because  3-D  terrain  is  supported  and  soil  properties 
can  be  specified  and  simulated  for  varying  climate.  GIS  data  has  not  been  imported  until 
recently  and  is  still  under  development. 

Organization 

K 

Objective 

Organization  K  meets  Objective  because  3-D  terrain  is  supported  and  provides  soil  data 
in  look  up  tables  and  is  defined  per  USCS  standards.  GIS  data  can  be  imported  via 
several  methods  using  third-party  software. 

Organization 

L 

Below 

Threshold 

Organization  L  is  Below  Threshold  because  no  capability  for  3-D  terrain  exists  and 
cannot  import  GIS  data.  Organization  L’s  response  was  vague  on  soil  properties  but  can 
be  classified  per  USCS  standards.  No  mention  of  climate  effects  and  if  they  can  be 
modeled. 

Worldwide  availability  was  scored  based  on  whether  or  not  the  vendor  had  the  resources  to  support  sales 
NATO  countries  and  worldwide.  This  included  multidisciplinary  staff  to  meet  the  demands  of  a  wide  range  of 
customers  in  a  wide  range  of  geographic  areas.  Would  this  affect  updates  to  the  software  and  the  update 
distribution  worldwide? 


Table  10-17.  NATO  Specific  Applications  -  Tool  Support 


NATO  Specific  Applications  -  Tool  Support 

Organization 

A 

Objective 

Organization  A  meets  Objective  and  currently  supports  a  worldwide  customer  base 
spanning  the  NATO  countries. 

Organization 

B 

Objective 

Organization  B  meets  Objective  stating  it  will  travel  worldwide  to  provide  on-site 
technical  training  and  support. 
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NATO  Specific  Applications  -  Tool  Support 

Organization 

C 

Objective 

Organization  C  meets  Objective  and  will  travel  worldwide  to  provide  on-site  technical 
training  and  support. 

Organization 

D 

Below 

Threshold 

Organization  D  is  Below  Threshold  because  it  does  not  consider  itself  a  commercial 
operation  and  there  is  no  formal  training  program  or  support.  They  do  however,  invite 
guests  to  their  site  for  collaborative  efforts. 

Organization 

E 

Threshold 

Organization  E  meets  Threshold  because  the  number  of  trainers  and  locations  would 
probably  have  to  be  updated,  potentially  by  “training  the  trainers,”  but  ean  be  achieved 
in  a  relatively  short  time  frame. 

Organization  E 
Objective 

Organization  E  meets  Objective  because  it  has  regional  offices  worldwide  and  offers  on¬ 
site  training  at  the  customer’s  facilities. 

Organization 

G 

Objective 

Organization  G  meets  Objective  because  it  has  regional  offices  worldwide  and  offers 
on-site  training  at  the  customer’s  facilities. 

Organization 

H 

Threshold 

Organization  H  meets  Threshold  because  it  can  provide  support  worldwide  but  the 
additional  cost  will  be  charged  to  the  customer. 

Organization  I 
Objective 

Organization  I  meets  Objective  because  it  has  regional  offices  worldwide  and  offers  on¬ 
site  training  at  the  customers  facilities. 

Organization  J 
Objective 

Organization  J  meets  Objective  because  it  has  offices  around  the  globe  that  support  both 
technical  and  training  needs. 

Organization 

K 

Below 

Threshold 

Organization  K  is  Below  Threshold  since  only  automated  support  is  offered. 
Representatives  are  not  physically  present  worldwide. 

Organization 

L 

Below 

Threshold 

Organization  L  is  Below  Threshold  because  support  will  only  be  provided  via  phone  or 
email.  Representatives  are  not  physically  present  worldwide. 

Worldwide  tool  support  was  examined  referencing  the  software’s  long  term  availability,  will  this  candidate 
have  the  ability  to  support  and  maintain  NATO  specific  modeling  events  for  7-12  years  after  implementation. 
The  licensing  structure  and  track  record  of  each  vendor  was  also  examined  to  see  how  information  would  be 
secured  and  firewalled  during  validation  efforts. 


Table  10-18.  NATO  Specific  Applications  -  Worldwide  Tool  Availability  to  Approved  Sources 


NATO  Specific  Applications  -  Worldwide  Tool  Availability  to  Approved  Sources 

Organization 

A 

Objective 

Organization  A  meets  Objective  because  they  are  capable  of  supporting  their  product  for 
up  to  20  years  with  worldwide  representation  and  established  firewall  protocol. 

Organization 

B 

Objective 

Organization  B  meets  Objective  because  they  are  capable  of  supporting  their  product 
long  term  with  a  worldwide  customer  base  and  provide  proven  firewall  protocol  to 
support  a  large  customer  base. 
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NATO  Specific  Applications  -  Worldwide  Tool  Availability  to  Approved  Sources 


Organization 

C 

Objective 

Organization  C  meets  Objective  by  providing  guaranteed  support  for  7-12  years  with 
flexible  licensing  options  for  company  growth.  Organization  C  operates  worldwide  with 
commercial  and  military  customers  utilizing  firewall  protocols. 

Organization 

D 

Below 

Threshold 

Organization  D  is  Below  Threshold  as  it  provides  daily  build  and  release  cycles  of  the 
software.  This  appears  to  be  problematic  from  a  security/firewall  aspect  and  also  with 
respect  to  standard  NATO  events. 

Organization 

E 

Objective 

Organization  E  meets  Objective  citing  that  industry  is  increasingly  using  Organization 
E’s  software  for  model-based  development,  specifically,  many  automotive  companies, 
such  as  Audi,  BMW,  Daimler,  Eord,  Toyota,  Volvo  and  VW.  Large  worldwide  user 
base  with  successful  firewall  capabilities. 

Organization  E 
Objective 

Organization  E  meets  Objective  because  it  has  regional  offices,  a  large  customer  base 
and  can  provide  long  term  support  for  their  product.  Organization  E  is  also  firewall 
capable. 

Organization 

G 

Objective 

Organization  G  meets  Objective  saying  it  has  network  licenses  or  node -locked  options 
available.  Software  is  firewall  capable,  and  serves  a  large  customer  base.  They  can 
provide  long  term  support. 

Organization 

H 

Threshold 

Organization  H  meets  Threshold  and  can  support  long  term  if  needed,  but  the  additional 
cost  will  be  charged  to  the  customer. 

Organization  I 

Above 

Threshold 

Organization  I  is  Above  Threshold  because  they  can  provide  long  term  support  in 
excess  of  20  years.  Each  piece  of  software  is  typically  node -lock  licensed  with  firewall 
capabilities  but  this  is  not  described  in  any  further  detail  by  Organization  1. 

Organization  J 
Objective 

Organization  J  meets  Objective  because  it  has  been  in  business  for  over  30  years  and 
can  continue  to  provide  long  term  support.  Currently  supporting  thousands  of  users 
utilizing  firewall  protocols  without  interruption. 

Organization 

K 

Threshold 

Organization  K  meets  Threshold  because  they  can  provide  long  term  support  for  the 
next  7-12  years.  Eirewall  protection  is  limited,  a  by-product  of  its  open  licensing 
structure. 

Organization 

L 

Below 

Threshold 

Organization  L  is  Below  Threshold  stating  that  the  licensing  agreement  is  optional  but 
can  be  used  under  a  license  agreement  if  agreed  upon.  Customer  base  and  firewall 
precautions  are  limited. 
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10.5.5  Final  Scoring 


The  weights  given  in  Table  10-1  MOE  and  MOP  Weighting  are  used  with  the  scores  for  the  individual  MOPs  discussed  above  to  combine  the  results 
into  a  single  weighted  average  score  for  each  Organization  as  shown  below  in  Table  10-19. 


Table  10-19.  Final  Weighted  Scores. 


MOE 

MOP 

A 

B 

C 

D 

E 

F 

G 

H 

I 

J 

K 

L 

Physics  Based 

0.75 

0.85 

0.50 

0.50 

0.70 

0.75 

0.85 

0.59 

0.37 

0.85 

0.85 

0.59 

Accuracy/ 

Validation  through  measurement 

0.75 

0.85 

0.70 

0.59 

0.79 

0.37 

0.79 

0.59 

0.59 

0.85 

0.75 

0.37 

Robustness 

Supports  time  and  frequency 
domain  analysis 

0.85 

0.85 

0.70 

0.85 

0.75 

0.75 

0.85 

0.59 

0.37 

0.75 

0.70 

0.00 

Template  based 

0.85 

0.79 

0.75 

0.63 

0.85 

0.75 

0.85 

0.16 

0.16 

0.85 

0.75 

0.16 

Flexibility 

Wheeled  or  tracked  vehicles 

0.37 

0.85 

0.73 

0.00 

0.63 

0.73 

0.77 

0.00 

0.37 

0.85 

0.77 

0.37 

Automotive  Subsystems 

0.73 

0.73 

0.52 

0.52 

0.73 

0.69 

0.77 

0.37 

0.52 

0.80 

0.69 

0.00 

Cost,  Maintenance, 
and  Training 

License 

0.68 

0.85 

0.81 

0.35 

0.43 

0.39 

0.78 

0.78 

0.00 

0.78 

0.77 

0.00 

Run  Time 

0.79 

0.75 

0.79 

0.55 

0.79 

0.83 

0.85 

0.55 

0.61 

0.85 

0.79 

0.48 

Training 

0.71 

0.68 

0.85 

0.50 

0.37 

0.74 

0.85 

0.42 

0.82 

0.70 

0.34 

0.82 

NATO  specific 

applications 

Supports  unique  terrain  or  mission 
definition 

0.73 

0.85 

0.73 

0.63 

0.37 

0.73 

0.85 

0.77 

0.77 

0.77 

0.85 

0.50 

Worldwide  tool  availability  to 
approved  sources 

0.85 

0.85 

0.85 

0.50 

0.85 

0.85 

0.85 

0.50 

0.85 

0.85 

0.50 

0.50 

Worldwide  tool  support 

0.85 

0.85 

0.85 

0.50 

0.70 

0.85 

0.85 

0.50 

0.85 

0.85 

0.50 

0.50 

Weighted  Average  Scores 

0.69 

0.83 

0.69 

0.45 

0.67 

0.68 

0.82 

0.42 

0.45 

0.82 

0.74 

0.34 
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10.6  ADDITIONAL  QUESTIONS  IDENTIFIED  DURING  AVT  MEETING  IN 
POLAND 

At  the  conclusion  of  the  review  and  discussion  of  the  information  submitted  in  response  to  the  RFI,  some 
additional  questions  were  posed  hy  members  of  the  committee.  The  intent  of  these  additional  questions  was 
to  clarify  in  more  specific  terms  how  well  the  various  tools  could  deal  with  the  deformable  soil  conditions, 
how  efficiently  the  codes  might  be  able  to  run,  and  whether  the  reaction  of  the  soil  was  part  of  the  core 
simulation  or  if  a  “co-simulation”  approach  was  used.  The  additional  questions  posed  are  shown  below. 

1 .  How  is  the  vehicle  to  soil  interaction  simulated  for  off-road  operations  within  your  solution? 

2.  Is  the  vehicle  system  formulated  in  multibody  dynamics  code  or  in  finite  element  code?  Is  the 
simulation  of  the  vehicle  run  separately  from  the  vehicle  to  soil  interaction  or  does  a  “co-simulation” 
process  exist? 

3.  Does  your  solution  utilize  a  classical  terramechanics  approach  (Bekker-Wong)  or  does  your  solution 
utilize  an  alternative  approach  such  as  discrete  elements  or  finite  element  analysis?  A  description  of 
your  methodology  would  be  helpful  and  if  already  submitted,  could  you  reattach  specifically  to  your 
response  for  the  purpose  of  clarification? 

4.  How  do  the  vehicle  model  and  the  soil  model  interface  during  the  simulation? 

5.  How  has  your  solution  been  made  available  for  commercial  use  (e.g.,  soft  soil  applications  for 
agriculture  or  heavy  earth  moving  or  other?)  Do  you  have  a  special  designation  or  name  for  this 
particular  simulation  solution? 

6.  Have  you  previously  validated  your  soft  soil  model  through  physical  test  and  if  so  when  did  this 
occur?  How  widely  distributed  within  the  commercial  or  government  user  market  is  your  soft  soil 
simulation  solution? 

Because  of  the  limited  amount  of  time  provided  to  the  organizations  to  develop  a  response  and  the  fact  that 
follow-up  questions  and  explanations  had  to  be  limited  due  to  time  constraints,  a  simpler  scoring  methodology 
was  utilized.  The  criteria  for  meeting  an  A  through  D  level  response  was  developed  and  the  various  responses 
were  scored  accordingly.  The  criteria  and  results  are  shown  below. 
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Table  10-19.  Additional  Questions  Scoring  Criteria 


1.  How  is  the  vehicle  to  soil  interaction  simulated 
for  off  road  operations  within  your  solution? 

2.  Is  the  vehicle  system  formulated  in 
multibody  dynamics  code  or  in  finite  element 
code?  Is  the  simulation  of  the  vehicle  run 
separately  from  the  vehicle  to  soil  interaction 
or  does  a  co-simulation  process  exist? 

A 

Complete  technical  response  explaining  approach 
to  vehicle  soil  interaction  identifying  approach 
beyond  Bekker-Wong  and  referencing 

information  provided  within  RFI 

Vehicle  and  terrain  fully  integrated  approach. 
Co-simulation  discussed  where  appropriate. 
Fully  integrated  physics-based  discussion  of 
multibody  vehicle,  flexible  body  and  soil 
interaction,  reference  to  both  time  and 
frequency  domain. 

B 

Methodology  referenced  but  not  completely 
explained.  Approach  more  vague  but  includes 
explanations  involving  FEA,  DEM,  or  other 
more  physics-based  approach  to  soil  mechanics, 
sinkage  and  soil  shoving  approaches  described 

Examples  of  co-simulation  or  integrated 
simulation  provided  including  multi  body 
dynamics.  Fewer  details  or  examples  provided. 
Some  work  in  progress  referenced  and  solution 
not  complete 

C 

Explanation  and  methodology  limited  to  Bekker- 
Wong  or  use  of  a  combination  of  empirical  and 
other  traditional  soil  mechanics  relationships 
(Janosi-Hamamoto)  Explanation  of  physics- 
based  approach  is  very  limited  and  is  not  clearly 
defined  or  solution  is  referenced  as  provided  by 
another  source  for  soil  mechanics.  DEM  or  other 
more  detailed  representations  not  provided 

Separate  codes  utilized.  Integration  or 

interaction  of  codes  not  fully  described.  MBD 
integrated  with  other  tire  or  track  terrain 
interface  models.  Ability  to  maintain  full 
dynamic  interaction  between  MBD  vehicle 
system  and  terrain  not  completely  explained 

D 

Vague  or  incomplete  response.  Capability  not 
developed 

Vague  or  incomplete  response.  Capability  not 
developed 
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3.  Does  your  solution  utilize  a  classical 
terramecbanics  approach  (Bekker-Wong)  or  does 
your  solution  utilize  an  alternative  approach  such 
as  discrete  elements  or  finite  element  analysis? 
A  description  of  your  methodology  would  be 
helpful  and  if  already  submitted,  could  you 
reattach  specifically  to  your  response  for  the 
purpose  of  clarification? 

4.  How  do  the  vehicle  model  and  the  soil 
model  interface  during  the  simulation? 

A 

DEM,  EEA  or  other  physics-based  approach 
described.  Soil  variables  accounted  for, 

examples  of  dynamic  sinkage  and  terrain  soil 
interaction  provided. 

Clear  description  of  the  methodology  utilized 
to  integrate  vehicle  and  soil  interaction. 
Examples  provided. 

B 

Description  of  methodology  not  complete  but 
expanded  beyond  traditional  Bekker-Wong. 
Integration  of  component  models  with 
deformable  soil  representations  described.  DEM 
in  progress  but  not  fully  developed  or  released. 
EEA  methods  described.  Methods  not  applied  to 
both  vehicle  types  (tracked  and  wheeled)  but 
work  in  progress. 

Methodology  not  as  well  defined.  Generic 
examples  provided  or  identified  as  work  in 
progress.  Actual  tire  to  soil  or  track  to  soil 
dynamics  and  resulting  soil  deformation  or 
load  reaction  not  as  well  defined  but  discussed. 

C 

Only  provides  Bekker-Wong  or  traditional 
VCI/RCI  parameters  from  NRMM.  Physics- 
based  soil  interaction  not  well  explained  or 
references  as  potential  work  in  progress  for  the 
future. 

Solution  explained  in  relatively  simple  terms  or 
identified  as  using  Bekker-Wong  or  other 
traditional  (Janosi-Hamamoto)  relationships. 
Empirical  relationships  or  look  up  tables 
identified  from  other  soil  dynamics  criteria. 
Soil  strength  variables  and  interaction  with  tire 
or  track  contact  points  not  well  defined. 
Dynamic  shear  response  not  fully  explained. 

D 

Terramecbanics  capability  not  well  explained, 
vague  references  to  Bekker-Wong  or  existing 
NRMM  tools. 

Vague  or  incomplete  response.  Capability  not 
developed. 
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5.  How  has  your  solution  been  made  available 
for  commercial  use  (i.e.,  soft  soil  applications  for 
agriculture  or  heavy  earth  moving  or  other?)  Do 
you  have  a  special  designation  or  name  for  this 
particular  simulation  solution? 

6.  Have  you  previously  validated  your  soft  soil 
model  through  physical  test  and  if  so  when  did 
this  occur?  How  widely  distributed  within  the 
commercial  or  government  user  market  is  your 
soft  soil  simulation  solution. 

A 

Tool  deployed  and  accepted  within  Industry  or 
Government.  Examples  of  users  provided 
relative  to  the  intended  use  of  Next  Generation 
NRMM. 

Validation  examples  provided  for  wheeled  and 
tracked  vehicles.  Discussion  of  intended 
upgrades  and  lessons  learned  based  on 
validation  efforts 

B 

Tool  partially  developed  or  deployed  to  other 
users.  Beta  sites  identified.  Ongoing  research 
and  investments  discussed  and  provided. 
Discussion  of  multi-platform  evaluations 
ongoing. 

Partial  validation  provided.  System  use  for 
prediction  purposes  and  prediction  of  fielded 
systems.  Developmental  examples  provided  or 
in  process.  Eull  vehicle  systems  identified 
including  correlation  to  test  results  such  as 
sinkage  or  tractive  effort  or  dynamic  response 

C 

Tool  only  deployed  in  an  R  and  D  or 
development  capacity,  only  used  by  provider  to 
support  development  contracts 

Validation  only  at  the  component  or  laboratory 
level.  Eull  system  validation  information  not 
provided.  Prediction  of  vehicle  performance 
correlated  with  actual  test  results  not  provided 

D 

No  Deployment  outside  of  provider,  no  example 
of  use  by  others  or  for  other  system  evaluation 
for  designated  customers 

No  validation  information  provided 
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Table  10-20.  Additional  Questions  -  Organizations  A  through  F 


Question 

Organization 

A 

Organization 

B 

Organization 

C 

Organization 

E 

Organization 

F 

1.  How  is  the  vehicle  to  soil  interaction 
simulated  for  off  road  operations  within  your 
solution? 

C 

A 

B 

C 

C 

2.  Is  the  vehicle  system  formulated  in  multibody 
dynamics  code  or  in  finite  element  code?  Is  the 
simulation  of  the  vehicle  mn  separately  from  the 
vehicle  to  soil  interaction  or  does  a  co-simulation 
process  exist? 

B 

A 

B- 

B- 

B 

3.  Does  your  solution  utilize  a  classical 

terramechanics  approach  (Bekker-Wong)  or  does 
your  solution  utilize  an  alternative  approach  such 
as  discrete  elements  or  finite  element  analysis? 
A  description  of  your  methodology  would  he 
helpful  and  if  already  submitted,  could  you 
reattach  specifically  to  your  response  for  the 
purpose  of  clarification? 

B- 

A 

C+ 

C- 

C 

4.  How  do  the  vehicle  model  and  the  soil  model 
interface  during  the  simulation? 

B- 

A 

B 

C 

A 

5.  How  has  your  solution  been  made  available 
for  commercial  use  (i.e.,  soft  soil  applications  for 
agriculture  or  heavy  earth  moving  or  other?)  Do 
you  have  a  special  designation  or  name  for  this 
particular  simulation  solution? 

C 

C 

B 

C 

A 

6.  Have  you  previously  validated  your  soft  soil 
model  through  physical  test  and  if  so  when  did 
this  occur?  How  widely  distributed  within  the 
commercial  or  government  user  market  is  your 
soft  soil  simulation  solution. 

C 

B- 

B+ 

D 

B+ 

Average  Grade 

C+ 

B-i- 

B 

C 

B 

Equivalent  Score 

0.73 

0.81 

0.76 

0.69 

0.77 
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Table  10-21.  Additional  Questions  -  Organizations  G  through  K 


Question 

Organization 

G 

Organization 

H 

Organization 

I 

Organization 

J 

Organization 

K 

1.  How  is  the  vehicle  to  soil  interaction 
simulated  for  off  road  operations  within 
your  solution? 

A 

B 

C 

A 

A 

2.  Is  the  vehicle  system  formulated  in 
multibody  dynamics  code  or  in  finite 
element  code?  Is  the  simulation  of  the 
vehicle  run  separately  from  the  vehicle  to 
soil  interaction  or  does  a  co-simulation 
process  exist? 

A 

B 

B 

A 

B 

3.  Does  your  solution  utilize  a  classical 
terramechanics  approach  (Bekker-Wong) 
or  does  your  solution  utilize  an  alternative 
approach  such  as  discrete  elements  or  finite 
element  analysis?  A  description  of  your 
methodology  would  be  helpful  and  if 
already  submitted,  could  you  reattach 
specifically  to  your  response  for  the 
purpose  of  clarification? 

B 

C 

C 

A 

A 

4.  How  do  the  vehicle  model  and  the  soil 
model  interface  during  the  simulation? 

A 

A 

C 

A 

A 

5.  How  has  your  solution  been  made 
available  for  commercial  use  (ie  soft  soil 
applications  for  agriculture  or  heavy  earth 
moving  or  other?)  Do  you  have  a  special 
designation  or  name  for  this  particular 
simulation  solution? 

A 

D 

A 

A 

B- 

6.  Have  you  previously  validated  your  soft 
soil  model  through  physical  test  and  if  so 
when  did  this  occur?  How  widely 

distributed  within  the  commercial  or 
government  user  market  is  your  soft  soil 
simulation  solution. 

B-i- 

C 

D 

C-i- 

C 

Average  Grade 

A- 

B- 

C-i- 

A- 

B-l- 

Equivalent  Score 

0.84 

0.74 

0.73 

0.84 

0.79 

10.7  SUMMARY  OF  RESULTS 

It  was  determined  that  currently  available  tools  exist  which  can  fill  most  of  the  committee  needs.  Many  of  the 
solutions  met  above  threshold  or  objective  levels  in  the  given  criteria  of  Accuracy,  Flexibility,  Cost,  and 
NATO  specific  applications. 

Accuracy  for  vehicle  system  performance  is  the  biggest  limitation  of  the  current  NRMM.  Validated  physics- 
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based  methods  will  potentially  be  an  improvement  over  the  current  empirical  methods  for  evaluating  original 
vehicle  and  suspension  designs.  Likewise  known  NRMM  shortfalls  with  tire  dynamics  and  soft  soil  behavior 
can  be  addressed  with  new  methods  and  be  in  a  position  to  meet  the  emergence  of  deformable  terrain  contact 
models. 

Additional  findings  show  that  industry  as  a  whole  is  providing  solutions  that  are  well  supported  not  only  in 
terms  of  technical  support  but  also  the  accessibility  to  support  with  many  organizations  boasting  a  worldwide 
presence.  This  increased  use  in  industry  has  led  to  broader  applications  such  as  robotics,  powertrain,  engine 
combustion,  aviation  industry,  etc.  This,  in  turn  has  created  a  substantially  increased  user  base  with  multiple 
users  at  each  site.  This  has  further  assisted  the  development  of  various  licensing  structures  that  allow 
streamlined  use  and  increased  firewall  protection  for  the  users  and  ultimately  decreased  costs.  Another  by¬ 
product  of  commercial  solutions  becoming  more  mainstream  over  the  past  few  decades  is  the  increased  ease 
of  use  by  implementing  more  template -based  solutions  and  additional  GUI  options  and  adaptations  as  opposed 
to  expert  user  requirements  noted  for  some  open  source  solutions.  This  increased  usage  and  worldwide 
support  also  equates  to  many  commercial  solutions  having  the  ability  to  support  NATO-specific  applications 
while  maintaining,  supporting,  and  protecting  NATO  members  who  are  users. 

Currently,  there  is  no  other  NATO  Government  approved  mobility  analysis  tool  solution  available.  As  noted 
above,  there  are  both  commercially  based  software  and  potentially  university  developed  (“open  source”) 
solutions  that  are  available  which,  based  on  the  information  submitted,  can  meet  the  needs  established  by  the 
committee  for  next  generation  NRMM.  Developing  a  new  start  solution  has  potential  drawbacks  as  seen  with 
the  current  NRMM,  particularly  as  it  relates  to  a  permanent  funding  and  organizational  support  effort.  A 
responsible  organization  will  help  to  address  some  of  the  issues  that  are  prevalent  now  such  as  various 
software  releases,  outdated  versions,  and  invalidated  add-on  modules  circulating  throughout  the  user 
community  (configuration  management).  This  will  constitute  the  need  for  a  continuous  funding  stream.  This 
then  benefits  the  user  community  with  up-to-date  software  versions  to  all  users,  consolidated  training  which 
insures  proper  use,  and  standardization  of  processes  and  data  formats  for  more  seamless  data  flow  within  the 
user  community.  The  committee  discussed  potential  funding  sources  and  the  effort  will  continue  to  solicit  and 
provide  that  funding  to  support  the  future  RTG  effort.  Before  this  can  be  implemented,  however,  there 
remains  significant  work  to  be  done  to  establish  appropriate  controls,  formats  and  validation  verification 
methodologies  to  approve  any  new  tool  and  insure  it  benefits  the  user  community.  The  current  priorities 
identified  in  the  initial  MOE/MOP  process  were  adequate  for  an  initial  query  of  industry  but  with  the  realized 
influx  of  information  and  the  knowledge  gained,  the  existing  MOE/MOP  may  need  to  be  reviewed  and 
updated.  Examining  items  such  as  mobility  as  a  survivability  enhancement  feature  is  emphasized  for  current 
and  future  vehicle  development. 


10.8  RECOMMENDED  NEXT  STEPS 

Continued  Evaluation  for  Validation 

As  discussed  in  the  summary  of  results,  it  is  apparent  that  the  multibody  dynamic  tools  which  are  available 
from  commercial  and  university  sources  are  capable  of  supporting  the  analysis  and  prediction  of  wheeled  and 
tracked  vehicle  systems  over  deformable  soil  conditions.  However,  the  focus  of  most  of  these  tools  has  been 
for  commercial  vehicle  system  development.  Many  of  the  potential  providers  are  not  fully  familiar  with  all  of 
the  capabilities  of  the  existing  NRMM,  particularly  as  it  relates  to  developing  specific  terrain  units  which  are 
appropriate  for  worldwide  deployment.  The  strength  of  the  tools  varies;  some  are  capable  and  have  been 
thoroughly  validated  for  on-road  operation  and  yet  only  limited  off-road  deformable  soils  work  has  been 
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accomplished.  Others  focused  primarily  on  off-road  soft  soil  terrain  but  have  no  capability  for  determining 
on-road  stability  and  associated  dynamic  control.  Ah  of  the  information  submitted  by  the  various 
organizations  in  response  to  the  RFI  had  very  limited  validation  and  verification  information.  In  some  cases 
this  was  due  to  the  fact  that  the  data  was  controlled  by  the  OEM  who  provided  all  of  the  vehicle  details;  in 
other  cases,  the  work  was  purely  theoretical  and  the  tools  had  not  been  compared  to  physical  results.  Some  of 
the  validation  was  conducted  on  events  which  are  not  representative  of  the  worldwide  deployment 
requirements.  For  this  reason  it  was  determined  that  additional  validation  and  verification  is  required  to  better 
quantify  the  functionality  of  the  various  tools. 

To  rapidly  complete  this  validation  effort  it  is  necessary  to  have  measured  vehicle  and  associated  test  data  to 
compare  against  the  predictions.  Theme  5  made  a  set  of  Recommendations  for  Benchmarking  the  tools 
described  above  to  Theme  7,  the  team  dealing  with  Verification  and  Validation.  Theme  5’s  recommendations 
are  contained  in  Appendix  F. 

10.9  CONCLUSIONS 

The  results  of  this  effort  indicate  that  a  variety  of  organizations  and  tools  exist  and  have  previously 
demonstrated  the  ability  to  accurately  simulate  complex  vehicle  system  performance  on  both  deformable  and 
non-deformable  surfaces.  Further,  data  exists  which  can  be  used  to  evaluate  and  validate  the  performance  of 
any  new  tool  set  while  including  the  latest  in  ground  vehicle  system  technology.  These  advances  are 
primarily  driven  by  investment  from  commercial  industry  and  are  focused  on  those  environments.  These 
results  demonstrate  that  it  will  not  be  necessary  to  initiate  a  new,  expensive,  and  time  consuming  development 
effort.  However,  because  the  needs  of  the  NATO  community  are  unique,  particularly  in  the  area  of  providing 
predictions  of  soft  soil  mobility  while  utilizing  temporal  environmental  information,  additional  investment  in 
the  validation  of  potential  tools  and  solutions  will  be  required. 

Existing  solutions  support  both  tracked  and  wheeled  vehicle  three-dimensional,  physics-based  multibody 
dynamic  analysis  and  therefore  it  is  anticipated  that  one  simulation  environment  can  provide  mobility  analysis 
for  combat  and  combat  support  vehicle  systems.  However,  recent  mobility  performance  data  for  new  vehicle 
systems  are  relatively  limited.  Therefore  investment  in  detailed  measurement  efforts  to  quantify  tire  or  track 
terrain  interface  in  order  to  support  the  tool  validation  process  should  be  anticipated. 

Tbe  validation  and  verification  next  step  effort  must  consider  tbe  vehicle  as  a  system  and  not  be  unnecessarily 
focused  on  the  tire  or  track  interface.  Suspension  and  powertrain  dynamics  which  provide  the  most  uniform 
ground  contact  pressure  and  uniform  power  delivery  have  demonstrated  best  soft  soil  mobility.  Success  of 
future  tools  will  be  dependent  upon  the  ability  of  these  tools  to  accurately  represent  the  environment  and  the 
vehicle  system  reaction  to  that  environment. 

As  noted  by  the  committee,  pure  mobility  measurement  over  a  homogenous  soil  represents  a  small  but 
important  part  of  the  current  NRMM  tool.  Predicted  speed  made  good,  dash  speed,  performance  over 
individual  terrain  units,  visibility,  etc.  are  all  aspects  of  the  current  NRMM  which  can  be  addressed  by  the 
future  MBD  tools.  As  noted  in  the  summary  of  results,  the  available  tools  are  affordable,  supported 
worldwide  and  are  able  to  quickly  complete  mobility  predictions  once  all  necessary  parametric  data  has  been 
input.  Revisiting  the  criteria  and  level  of  importance  for  each  of  the  evaluation  elements  throughout  the  next 
step  process  will  be  important  to  tbe  success  of  the  effort.  Continued  interaction  with  industry  has  verified 
that  physics-based  MBD  tools  exist  which  meet  the  various  criteria  including  affordability.  Furthermore  by 
implementation  of  multi-core  co-simulation  techniques,  industry  has  proven  that  high-speed  computing 
capability,  while  helpful,  may  not  be  essential.  Available  modularity  in  tbe  various  analysis  codes  has  helped 
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to  insure  necessary  flexibility  to  address  future  concepts  and  designs.  Next  step  determination  of  Verification 
and  Validation  techniques,  configuration  management,  software  release  version  management,  etc.  will  be 
essential  to  the  success  of  the  effort  given  the  substantial  increase  in  emphasis  on  enhanced  vehicle  mobility.  . 

Based  on  the  information  gathered  it  is  recommended  that  the  evaluation  process  continue  as 
replacement/update  of  the  current  NRMM  is  critical.  Knowledge  of  geotechnical  properties  and  knowledge  of 
vehicle  system  properties  including  electronic  controls  will  be  essential  to  the  success  of  the  effort. 
Substantial  additional  funding  requirements  are  anticipated  to  support  this  more  detailed  validation  and 
verification  effort.  It  is  recommended  that  a  tiered  approach  be  taken,  evaluating  potential  solutions  against 
the  relatively  simpler  events  and  then  including  the  more  challenging  soft  soil  traction,  turning,  obstacle 
avoidance,  and  negotiation  events.  It  is  recommended  that  worldwide  events,  significant  to  the  various 
countries  and  operational  environments  be  included.  Based  on  the  current  participation  and  capabilities  within 
the  committee  the  following  support  could  be  considered. 

Road  roughness  -  Conditions  in  Turkey  run  the  gamut,  from  original  stone  roads  from  Roman  times  to  the 
most  advanced  highway  system  technology.  Substantial  investment  and  knowledge  of  these  conditions  and 
use  of  that  data  will  help  insure  a  representative  and  robust  solution  for  the  broad  range  of  road  and  trail 
roughness. 

Environmental  variables  -  USA  CRREL  has  spent  many  years  in  the  study  of  erosion,  freeze  thaw  impacts  on 
soil  strength,  trail  roughness  measurement,  and  how  the  terrain  conditions  change  with  traffic.  This  input  will 
be  very  helpful  to  the  future  validation  process. 

Soft  soil  conditions  -  Estonia  -  Their  current  efforts  to  accurately  quantify  soil  type,  plastic  and  liquid  limit, 
impact  on  ground  bearing  strength,  correlation  to  ground  contact  pressure,  and  their  available  data  on  a  range 
of  load  and  tire  deflections  will  add  substantially  to  the  available  database.  This  support  can  be  used  for  both 
input  to  simulations  and  for  validation  purposes. 

Impact  on  mobility  and  soil  strength  as  a  function  of  vegetation  spacing,  root  structure,  and  demands  on 
maneuver  -  Czech  Republic  -  Their  significant  studies  on  the  impact  of  vegetation  on  soil  strength  and 
structure,  and  thus  the  demands  on  vehicle  tractive  effort  and  uniform  ground  contact  pressure,  can  provide 
essential  measurement,  test,  and  validation  data  in  support  of  next  generation  tool  evaluation. 

Overarching  application  of  next  generation  tool  -  Canada  -  Their  current  work  in  evaluation  of  both  single 
and  multiple  vehicle  system  performance  and  identification  of  critical  output  elements  for  the  purpose  of 
vehicle  capability  evaluation  and  comparison  will  be  essential  toward  the  future  tool  development. 

Application  of  alternative  metrics  -  Germany  -  The  limitations  of  single  axis  measurements  such  as  vertical 
absorbed  power  (6  Watt)  have  been  fully  recognized  and  as  such  Germany  has  implemented  alternative  ISO- 
based  dynamics  measurements  and  associated  simulation  development.  Such  a  three-dimensional  validation 
approach  to  account  for  the  performance  of  the  entire  vehicle  system  over  complex  terrain  will  be  essential  for 
the  success  of  the  next  generation  simulation  environment. 

Vehicle  dynamics  analysis  -  Denmark  -  Based  on  investment  in  vehicle  safety,  vehicle  handling,  and  surface 
to  vehicle  interaction,  their  support  to  properly  define  representative  events  for  vehicle  stability  and  control, 
validation  of  the  simulations  for  those  events,  and  the  integration  of  those  events  into  the  overall  mission 
profile  will  help  insure  that  the  final  next  generation  solution  will  successfully  address  vehicle  performance  on 
surfaces  with  low  coefficients  of  friction. 
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With  full  NATO  support  the  team  can  he  assembled  to  properly  evaluate  each  step  of  the  validation  and 
verification  process  and  can  insure  that  the  subsequent  tool  selection  can  successfully  meet  the  necessary 
range  of  conditions  for  worldwide  deployment. 
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Chapter  11  -  THEME  6:  INPUT  DATA  AND  OUTPUT  METRICS 

Brian  Wojtysiak 

11.1  GOALS  AND  DELIVERABLES 

The  goal  of  the  Input  Data  /  Output  Metric  suhcommittee  (Theme  6)  is  to  define  the  Input  /  Output  data 
requirements  that  will  inform  the  Next -Generation  NRMM  tool  development  /  selection  processes. 

The  Input  Data  /  Output  Metric  suhcommittee  (Theme  6)  intends  to  develop  the  following  set  of  deliverables 
including: 

•  A  list  of  important  NRMM  inputs  parameters  /  variables 

•  A  list  of  output  products  that  should  be  generated  by  the  Next-Generation  NRMM 

•  Identification  of  proper  data  resolution  levels  for  inputs  /  outputs 

•  Identification  of  any  potential  data  standards  (OGC  compliant) 

•  Identification  of  key  input  /  output  considerations  that  will  shape  /  affect  the  software  system  design 

11.2  INPUT  DATA  /  OUTPUT  METRIC  SUBCOMMITTEE  MEMBERSHIP 

On  August  26,  2014,  the  NATO  AVT  ET-148  Study  Leadership  established  the  Input  Data  /  Output  Metric 
subcommittee  (Theme  6);  and,  on  September  08,  2014,  asked  representatives  from  the  US  Army  Materiel 
Systems  Analysis  Activity  (AMSAA)  to  lead  it.  As  mentioned  above,  the  subcommittee  membership  (listed 
below)  was  asked  to  further  refine  the  Input  /  Output  requirements  that  were  derived  from  an  initial  NRMM 
Modernization  survey,  distributed  to  the  committee  membership,  which  solicited  feedback  on  the  positive  / 
negative  aspects  of  the  current  NRMM  and  areas  where  improvements  were  needed. 

The  theme  members  are  shown  below: 


Country 

Name 

Canada 

Mayda,  William 

Czech  Republic 

Rybansky,  Marian 

Estonia 

Vennik,  Kersti 

USA 

Gunter,  David 

USA 

Jayakumar,  Paramsothy 

USA 

Letherwood,  Michael 

USA 

Ngan,  James 

USA 

Shoop,  Sally 

USA 

Ward,  Derek 

USA 

Wojtysiak,  Brian:  Leader 
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11.3  INPUT  DATA  /  OUTPUT  METRIC  REFINEMENT  APPROACH  AND 
RESULTS 


In  preparation  for  discussions  at  the  NATO  meetings  in  Brussels,  Belgium  from  October  13-17,  2014,  the 
subcommittee  grouped  the  Input  /  Output  data  feedback  received  from  the  committee  into  four  (4)  main 
categories  of  data  that  loosely  correlate  with  the  existing  data  categories  utilized  within  the  NRMM  framework. 
These  data  categories  were: 

1.  Vehicle  Data 

2.  Terrain  Data 

3.  Environment  /  Scenario  Data 

4.  Operator  Data 

Over  the  course  of  the  ET,  these  data  categories  evolved  to  incorporate  three  (3)  additional  data  categories  (in 
addition  to  the  four  (4)  identified  above): 

5.  Human  Eactors  Data 

6.  Autonomous  /  Semi-Autonomous  Vehicle  Data 

7.  Scale  /  Resolution  Modes 

In  addition  to  capturing  the  types  of  data  needed  to  support  the  modeling  effort,  the  subcommittee  needed  to 
consider  other  critical  Input  /  Output  data  factors  including: 

•  Einding  a  balance  between  model  fidelity,  availability  of  required  input  data,  time  to  construct  model 
input  data  sets,  model  execution  runtimes,  and  desired  output  products 

o  The  model  must  be  able  to  model  everything  from  paper  concepts  to  detailed  engineering 
designs 

o  The  model  must  be  able  to  allow  for  quick  input  file  construction  (i.e.  willing  to  sacrifice 
some  fidelity  to  conduct  analyses  for  short  suspense  items) 
o  The  model’s  minimum  Input  data  requirements  must  consider  the  level  of  data  available  (at 
all  data  resolution  levels  throughout  the  system’s  development  /  acquisition  cycle) 

•  Incorporating  data  elements  needed  to  evaluate  “new”  vehicle  technologies  (i.e.  physical 
implementations,  control  systems,  autonomous  systems,  bipedal  movement) 

•  Enabling  the  Next-Generation  NRMM  to  handle  time-series  data 

•  Developing  mechanisms  for  updating  NRMM’s  “static”  terrain  libraries  to  reflect  new  operational 
areas  of  interest  /  evolving  terrain  conditions 

•  Identifying  what  terrain  response  characteristics  are  needed: 

o  Currently  NRMM  factors  in  deformable  soils,  snow  /  ice,  vegetation,  obstacles,  surface 
roughness,  amphibious  operations,  weather  effects 
o  “Non-traditional”  terrain  surfaces  (e.g.  robotic  platforms  -  carpet,  slate,  tile,  etc.) 

•  Identifying  what  improved  human  factors  representations  are  needed: 

o  NRMM  currently  considers  vibration  doses,  visibility,  response  times,  etc.  Are  there  others? 
o  Do  we  need  to  modify  any  of  these  approaches  (i.e.  vibration  dose  at  multiple  vehicle 
locations,  seated  vs.  supine  -  e.g.  casualty  evacuation)? 

•  Improving  User  Interface  /  Data  Validation  and  Error  Handling  to  ensure  erroneous  results  are  not 
inadvertently  generated  due  to  a  user’s  lack  of  familiarity  with  the  model  parameters  /  user  inputs 

•  Determining  the  modes  of  operation: 

o  Batch  vs.  Individual  runs 
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o  Real-time  vs.  Non-Real-time 

o  User  Experience  and  /  or  Role-Based  Interfaces  (Novice,  Intermediate,  Advanced  or 
Developer,  Practitioner,  Supervisor  /  Practitioner,  Novice  /  Operational  User) 

•  Defining  the  output  products  /  level  of  detailed  needed: 

o  Common,  easy  to  understand  metrics  for  leadership/stakeholders 

o  Detailed,  intermediate  metrics  (e.g.  reason  codes,  rut  depth,  overriding  forces)  for  the  subject 
matter  expert  to  provide  insights  on  final  results 

•  Defining  all  “potential”  mobility  metrics 

o  Current:  Trafficability  (GO/NOGO),  “speed  made  good,”  VCl 

o  Next-Gen:  Other  on-road  mobility  metrics  (e.g.  acceleration,  maneuvers)  applied  to  off-road 
performance;  path-finding;  operational  scenario  metrics  (e.g.  mission  time,  speed),  etc. 

•  Characterizing  uncertainty  associated  with  precision  of  model  input  data 

o  Stochastic  vs.  Deterministic  approach 

•  Reducing  time  /  effort  needed  to  summarize  results  into  products  that  are  easy-to-understand 

•  Ensuring  Next-Generation  NRMM  conforms  to  commercial,  military,  and  open  source  vehicle  and 
geospatial  analysis  data  standards  to  promote  data  interoperability  with  other  analysis  tools  /  data 
sources 

Eollowing  the  meeting  in  Brussels,  Belgium,  the  Input  /  Output  subcommittee  further  refined  the  Input  / 
Output  requirements  and  decomposed  the  Input  /  Output  data  categories  into  smaller  and  smaller  data 
elements  (e.g.  subsystems,  assemblies,  components,  data  elements). 


Eor  example,  the  Vehicle  Information  category  was  decomposed  into  smaller  data  segments  including: 

1.  Vehicle  Physical  Dimensions 

2.  Traction  Information 

3.  Driveline  Information 

4.  Suspension  Information 

5.  Multi- Axle  /  Multi-Unit  Considerations 

6.  Other 

Eollowing  this  step,  the  subcommittee  identified  the  data  elements  within  each  of  these  sub-classifications. 
Eor  example,  the  additional  data  deconstruction  for  the  Vehicle  Driveline  is  outlined  below: 

1 .  Driveline  Information 

a.  Engine  Parameters 

i.  Mass 

ii.  Moment  of  Inertia  (3  axes) 

iii.  Mounting  Locations 

iv.  Rotating  Mass  (Crankshaft)  Inertia 
V.  Mounting  Locations 

vi.  Mount  Stiffness  (Eorce  vs.  Displacement)  (all  directions) 

vii.  Damping  Eorce  vs.  Velocity 

b.  Power  /  Torque  Curves 
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c. 

Torque  Converting  Characteristics 

i. 

Mass 

ii. 

Moment  of  Inertia 

iii. 

Center  of  Gravity  Location 

iv. 

Locking  Logic 

d. 

Transmission  Characteristics 

i. 

Mass 

ii. 

Moment  of  Inertia 

iii. 

Center  of  Gravity  Location 

iv. 

Mounting  Location 

V. 

Mount  Stiffness  (Force  vs.  Displacement)  (all  directions) 

vi. 

Damping  Force  vs.  Velocity 

vii. 

Number  of  Gears  and  Ratios 

viii. 

Efficiency 

e. 

Shifting  Logic 

f 

Differential  /  Gear  Hubs 

i. 

Mass 

ii. 

Moment  of  Inertia 

iii. 

Center  of  Gravity  Location 

iv. 

Mounting  Location 

V. 

Mount  Stiffness  (Force  vs.  Displacement)  (all  directions) 

vi. 

Damping  Force  vs.  Velocity 

vii. 

Number  of  Gears  and  Ratios 

viii. 

Efficiency 

g- 

Hybrid  /  Electric  Powerplants  -  Regeneration 

h. 

Turning  Diameter  /  Skid  Steer 

i. 

Engine  Fuel  Map 

j- 

Engine  Cooling  Demands 

The  complete  decomposition  is  reflected  in  Tables  1 1-2  to  11-4  which  follow. 

A  similar  process  was  used  to  map  /  trace  the  inputs  to  the  output  products  /  decisions  supported.  The  Input  / 
Output  subcommittee  developed  an  initial  list  which  was  shared  and  vetted  with  the  NATO  AVT  ET-148 
membership  at  the  NATO  meeting  in  Rzeszow,  Poland.  The  final  list  of  Output  Products  /  Output 
Considerations  approved  by  the  membership  of  NATO  AVT  ET-148  is  captured  in  Table  11-5  below. 

Finally  the  Input  /  Output  subcommittee  generated  a  series  of  “Other  Data  Input  /  Output  Factors  to 
Consider”.  These  factors  include: 

1 .  Data  Availability 

2.  Data  Resolution  /  Scale 

3.  Customization  Capability 

4.  Stochastic  vs.  Deterministic 

5.  Open  Source/GOTS  vs.  Proprietary 
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6.  Future  Growth 

7.  Ease  of  Use  /  Reuse 

8.  Steady  State  vs.  Non-Steady  State  Behavior 

9.  Real-time  vs.  Non-Real-Time 

10.  Data  Standards 

1 1 .  Spatial  Data  Capahilities 

12.  User  Interface  -  GUI  /  Command  Line 

13.  Modes  of  Operation 

Each  of  these  “Other  Faetors”  are  explained  in  more  detail  in  Tablel  1-6  (which  follows). 
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Classification 

Parameters 

Used  to  determine 

GIS  applications 

Length,  Width,  Height,  Frontal  /  Side 
Profile 

Envelop  clearance  (tunnels,  bridges, 
overhead  wires...),  frontal  area 
(aerodynamics) 

Go  /  No-Go  constraints  for 
urban  terrain  mobility 
analysis 

Bottom  profile  (3  dimensional) 

Undercarriage  clearance 

Obstacle  Go  /  No-Go  Layer 

Clearance 

Undercarriage  clearance 

Obstacle  Go  /  No-Go  Layer 

c 

o 

'v5 

c 

0^ 

E 

o 

Hard  points  (e.g.  control  arms,  bump 
stops,  rebound  stops,  spring/ shock 
mounts,  tie  rod,  wheel  center,  drive 
shaft,  sub-frame,  anti-roll  bar,  spring 
lengths) 

Forces  acting  upon  components, 
deflection  of  components  under 

stress 

> 

Mass  /  Material  properties  (mass, 
material  strength,  eg  location, 
moments,  force  vs.  velocity  curves, 
forces  vs.  displacement  curves,  etc) 

Forces  acting  upon  components, 
deflection  of  components  under 

stress 

Pushbar  height  /  geometry  (i.e. 
frontal  area  -  CAD  representation?) 

Go  /  No-Go  in  vegetation  area 
(override  vegetation  force) 

Vegetation  Go  /  No-Go  Layer 

o 

•*- 

c 

!c 

Wheeled  vehicle:  Tire  size.  Outside 
Diameter,  rim  diameter,  deflection, 
rolling  radius,  ground  contact  area 
(tireprint),  number  of  axles,  number 
of  tires  peraxle  (dual,  single),  axle 
spacing,  tread  width,  tread  depth, 
track  width,  tire  inflation  pressure 
(static  vs.  dynamic  -  CTIS);  tire 
construction  materials;  tire  models 

Tire  factor,  speed  limitation  due  to 
tire  type,  VCI 

Tire  speed  limiter  layer.  Go  / 
No-Go  layer 

0^ 

> 

Tire  type:  Pneumatic  vs.  non¬ 
pneumatic;  type  bias  ply,  radial,  rigid, 
airless,  run-flat 

Tire  factor,  speed  limitation  due  to 
tire  type,  VCI 

Tire  speed  limiter  layer.  Go  / 
No-Go  layer 

Traction  Info 

Tracked  vehicle:  Track  length,  track 
width,  ground  contact  area,  grouser 
height  /  pitch,  track  shoe  area, 
roadwheel  spacing,  idler/ sprocket/ 
roadwheel  radius;  track  models;  track 

tension 

Track  factor,  ground  factor,  VCI 

Non-standard  vehicles:  Bi-pedal 
robots,  driven  wheel  hubs,  etc. 

Track  factor,  ground  factor,  VCI 

Slip  at  maximum  drawbar  pull  -  Mu 
slip  /  Mu  alpha  curve 

Tractive  effort 

Braking  coefficient  /  transmission 
retarder/ engine  braking 

Maximum  braking  force,  stopping 
distance  (No-Go  if  visibility  distance 
<  stopping  distance) 

Visibility 

CG  height  -  position  (x,  y,  z) 

Rollover  characteristics 

Right  track  /  Left  track  path  (e.g.  2D 
bicycle  model  to  3D  model) 

Ride  dynamics.  Vehicle  Trafficability 
(VCI) 

GVW,  CG  location  (height, 
longitudinal,  lateral).  Weight  per 
axle.  Spring  /  damping  characteristics 

Ride  dynamics.  Vehicle  Trafficability 
(VCI) 

Ride  dynamic  speed  iimit 
layer/  Go  /  No-Go  layer 

Table  11-2:  Vehicle  Information  Parameters  (Dimensions,  Traction  Information)  (1  of  2) 
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Vehicle  Info 


Classification 

Parameters 

Used  to  determine 

GIS  applications 

Engine  parameters  (mass,  moment  of  inertia  (3  axes), 
rotating  mass  (crankshaft)  inertia,  mounting  locations, 
mount  stiffness  (force  vs.  displacement)  (all  directions), 
damping  (force  vs.  velocity) 

Tractive  effort 

Power /  torque  curves 

Tractive  effort 

Tractive  effort  Go  /  No-Go 
layer 

Torque  converter  characteristics  (e.g.  mass,  moment,  eg 
location)  /  locking  logic 

Tractive  effort 

Tractive  effort  Go  /  No-Go 
layer 

Transmission  characteristics:  mass,  moments,  eg 
location,  mounting  locations,  mount  stiffness  (force  vs. 
displacement)  (all  directions),  damping  (force  vs. 
velocity),  number  of  gears  and  ratios,  efficiency 

Tractive  effort 

Tractive  effort  Go  /  No-Go 
layer 

Driveline 

Shifting  logic 

Tractive  effort. 

Tractive  effort  Go  /  No-Go 

Info 

Fuel  Performance 

layer 

Differential  /  gear  hubs;  mass,  moments,  eg  location, 
mounting  locations,  mount  stiffness  (force  vs. 

Tractive  effort. 

Tractive  effort  Go  /  No-Go 

displacement)  (all  directions),  damping  (force  vs. 
velocity),  number  of  gears  and  ratios,  efficiency 

Fuel  Performance 

layer 

Hybrid  /  Electric  Power  Plants  -  Regeneration 

Tractive  effort. 

Fuel  Performance 

Tractive  effort  Go  /  No-Go 
layer 

Urban 

Maneuverability 

Go  /  No-Go  constraints  for 

Turning  Diameter/ Skid  Steer 

urban  terrain  mobility 
analysis 

o 

c 

Engine  Fuel  Map 

Fuel  Performance 

0) 

u 

Ic 

Engine  Cooling  Demands 

Degradation  in 
Tractive  Effort 

Tractive  effort  Go  /  No-Go 
layer 

Qi 

> 

Suspension 

Info 

Subsystem  Characteristics 

Ride  dynamics. 
Vehicle 

Trafficability  (VCI) 

Ride  dynamiespeed  limit 
layer  /  Go  /  No-Go  layer 

Multi-axle  / 
Multi-unit 

Info 

Trailers,  multiple  steered  axles,  tandem  trailers,  etc. 

Dynamics  / 
Maneuverability 

Drawbar,  rolling  resistance 

Tractive  effort 

Tractive  effort  Go  /  No-Go 
layer 

Parasitic  power  losses  -  cooling  fans,  vehicle 
electronics,  etc. 

Loss  of  propulsion 
power  -  reduced 
tractive  effort 

Tractive  effort  Go  /  No-Go 
layer 

Other 

Control  logic  -  Electronic  Stability  Control  /  Traction 
Control  /  Anti-Lock  Braking/ Active  and  Semi-Active 
Suspension  Systems 

Vehicle 

intervention  to 

maintain  stability 
/  control 

Environmental  factors  -  (e.g.  hot  vs.  cold  effects) 

Loss  of  propulsion 
power  -  reduced 
tractive  effort 

Tractive  effort  Go  /  No-Go 
layer 

Vehicle 

Operation  with  degraded  state 

Trafficability  (VCI), 
Speed  limiter 

All  GIS  layers 

Swimming  /  fording  speeds 

Go/No-Go  in  water 

Water  bodies  Go  /  No-Go 
layer 

Table  11-2:  Vehicle  Information  Parameters  (2  of  2) 


(Driveline,  Suspension,  Multi- Axle  /  Multi-Unit,  Other) 
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Terrain 


Classification 

Parameters 

Used  to  determine 

GIS  applications 

Terrain 

Spatial 

Orientation 

Spatial  orientation  of  data  (lat/long,  MGRS,  etc),  vector 
feature  data  (point,  lines,  polygons),  rasterdata  (DIED, 
LIDAR,  etc).  Compliant  with  GIS  data  standards 

Spatial  capabilities.  Ability  to  quickly 
/  easily  update  terrain  data 

All  GIS  layers 

Off-road 

Surface  slope  (%) 

Slope  resistance 

Slope  Go/ No-Go  Layer 

Surface  materials  (soil  type,  soil  classification  system, 
soil  moisture),  soil  cohesion,  snow  depth /density,  soil 
strength  (RCI,  Cl),  hard  surface  rolling  resistance,  soil 
sinkage,  soil  compaction /density,  frost /thaw depth, 
split  mu  -  gravel  shoulder,  road  edge,  surface  material 

reflectance 

Soil  resistance,  VCI  (FGS,  CGS, 
Muskeg)  -  bearing  capacity  /  sheer 
strength,  reflectance  affects 
autonomous  sensing  capabilities 

Soil  strength  Go  /  No-Go  Layer 

Surface  roughness 

Go/  No-Go  area,  speed  limiter 

Ride  dynamicspeed  limit  layer 

Natural  obstacles:  cliffs,  ridges,  trenches,  mounds, 
embankment  climbing, ... 

Go/  No-Go  area,  speed  limiter 

Obstacle  Go  /  No-Go  Layer, 
Maneuverability  layer. 
Amphibious  Egress  Locations 

Man  made  obstacles:  cuts,  pipe  lines,  rubble  piles 

Go  /  No-Go  due  to  obstacles 

Obstacle  Go  /  No-Go  Layer, 
Maneuverability  layer 

Non-standard  terrain  surface  materials:  friction  co¬ 
efficients  /  rolling  resistances  for  surfaces  such  as  tile, 
carpet,  slate  floors,  etc. 

Go/  No-Go  area,  speed  limiter 

Ride  dynamicspeed  limit 
layer.  Obstacle  Go  /  No-Go 

Layer 

Vegetation,  stem  size,  stem  spacing 

Go  /  No-Go  due  to  vegetation 

Vegetation  Go  /  No-Go  layer. 
Maneuverability  layer 

Water  bodies:  lakes,  ponds,  oceans,  streams,  surf  zones, 
drainage  (rivers,  canals),  velocity  of  flowing  water 

Go/  No-Go,  speed  limiterdue  to 
waterbodies 

Limit  accessible  area.  Water 
Go/ No-Go  Layer 

Numberof  vehicle  passes  (e.g.  Vlvs.  V50) 

Go/ No-Go  limiter 

Limit  accessible  area 

Railroad  tracks 

Limit  accessible  area 

Limit  accessible  area 

On-road 

Road  superelevation  angle 

Sliding,  tipping,  rollover 

Urban  mobility 

Road  width 

Go/  No-Go  in  urban  terrain 

Urban  mobility 

Surface  type  /  roughness  coefficient 

Speed  limiter 

Ride  dynamicspeed  limit  layer 

Road  radius  of  curvature 

AASFITO  curvature  speed  limit, 
sliding,  tipping,  rollover 

Urban  mobility 

Infrastructure  Limitations  -  Military  Load  Classification 
of  Bridges,  pavement  weight  capacity  limits,  etc 

Go/ No-Go  limiter 

Limit  accessible  area 

Overhead  (overpass,  wire,  bridge) 

Go  /  No-Go  due  to  overhead 

clearance 

Urban  mobility 

Scenario 

Snow  covered,  ice  covered  roads 

On  road  surface  traction  condition 

Day  /  Night 

Visibility/Sensor  performance 

Dry,  Wet,  Wet-Wet,  Snow,  Sand,  Fog 

Soil  strength  per  operating  scenario. 
Visibility 

Soil  Go /No-Go  layer 

Table  11-3:  Terrain  /  Scenario  Parameters 
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Classification 

Parameters 

Used  to  determine 

GIS  applications 

Human  Factors 

Ride  /  shock 

Speed  limitation  due  to  "comfort" 

Ride  dynamic  speed  limit 
layer 

Multiple  ride  /  shock  locations 

Speed  limitation  due  to  "comfort"  - 
e.g.  driver  seat  and  MEDEVAC  litter 

Ride  dynamic  speed  limit 
layer 

Eye  height 

Go  /  No-Go,  Visibility  controlling 
speed  for  each  slope 

Operator  Behavior 

Path  /  Line  Selection  (requires  time 
series  capability)  /  driver  model 

Dynamics  /  Maneuverability,  Sliding, 
tipping,  rollover 

Visibility 

Speed  limiter 

Response  time  (e.g.  braking) 

Speed  limiter 

Human-in-the-loop  feedback 

Dynamics  /  Maneuverability,  Sliding, 
tipping,  rollover,  speed  limiter 

Non-steady-state  behavior  (e.g. 
acceleration  /  deceleration,  steering 
inputs,  etc.) 

Dynamics  /  Maneuverability,  Sliding, 
tipping,  rollover 

Autonomous 

Semi-Autonomous  Vehicles 

Situational  Awareness  -  Sensor 

Height,  Sensor  Range,  Sensor 
Resolution,  GPS  location,  GPS  error, 
inertial  navigation  schema,  inertial 
navigation  limits 

Ability  to  sense  environment 

Autonomy  Level  -  full,  teleoperation, 
semi-autonomous,  shared  control, 

none,  etc. 

Ability  to  remotely  communicate  / 
operate  system  remotely 

A-priori  terrain  knowledge 

Ability  to  navigate  /  respond  to 
environmental  stimuli 

Decision  logic/ control  systems 

How  the  system  will  respond  to 

environmental  stimuli 

Constrained  by  Traffic  rules  (lanes, 
signals,  speed  limits) 

How  the  system  will  respond  to 

environmental  stimuli 

Performance  limits  (e.g.  vibration 
levels  to  prevent  damage  to 
electroniccircuitry/ sensor 
degradation,  temperature  /  humidity 
effects,  slippage,  balance  /  stability 
issues,  etc) 

Speed  limitation  due  to  "comfort", 
performance  degradations 

Ride  dynamic  speed  limit 
layer 

Performance  limits  associated  with 
any  payloads  -  (i.e.  vibration  limits  for 
sensor  suites,  munitions,  etc) 

Speed  limitation  due  to  "comfort", 
performance  degradations 

Ride  dynamic  speed  limit 
layer 

Teleoperation  -  RF  communication 
capability,  latency  /  lag  time  in 
communication  between  system  / 
operator;  Use  of  pre-determined 
"waypoints",  human-in-the-loop 
inputs;  bandwidth  /  spectrum 
limitations 

Ability  to  remotely  communicate  / 
operate  system  remotely 

Scale  /  Resolution  Modes 

System  level.  Subsystem  Level, 
Component  Level 

Ability  to  support  all  data  fidelity 

levels 

Empirical  Soil  /  Detailed  Soil  (Physics- 
based) 

Ability  to  support  all  data  fidelity 

levels 

Table  11-4:  Humans  Factors,  Operator  Behavior,  Autonomous  /  Semi- Autonomous  and  Scale  / 


Resolution  Parameters 
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Item  # 

Output  Products  /  Output  Considerations 

1 

Cartographic  Map  products  and  /  or  spatially-oriented  data  that  can  be  imported 
into  a  CIS  visualization  tool  (OGC  /  Military  Compliant) 

2 

Speed  comparisons  between  vehicles /Top  Speed 

3 

Trafficability  comparisons  between  vehicles 

4 

No-Go  /  Speed  limiting  reason  codes 

5 

Vehicle  stability  /  handling  results  -  lateral  acceleration,  static  /  roll  stability,  etc. 

6 

Urban  Maneuverability  Modeling 

7 

Path  Modeling 

8 

Obstacle  Negotiation 

9 

Backward  compatibility  to  previous  NRMM  model  (VCI  /  RCI) 

10 

Fuel  Consumption  /  Economy 

11 

Vehicle  Range 

Acceleration  /  Deceleration  Characteristics 

13 

Separate  On-road  vs.  Off-Road  Performance  Summary 

14 

Minimize  Effort  Required  to  Post-Process  Model  Results  into  Analytical  Products 

15 

Multiple  output  product  levels  -  operational,  engineering-level,  etc. 

m 

Spatial  analysis  considerations  in  result  generation  (e.g.  elimination  of  spatial  No- 
Go  "islands") 

mm 

Uncertainties  associated  with  Output  Values 

18 

Powertrain  and  braking  torque  applied  at  each  traction  element  (e.g.  wheel, 
track  element) 

19 

Buoyancy  /  Amphibious  Speed 

20 

Ride  Quality  /  Absorbed  Power 

21 

Minimum  Turning  Radius  -  wall-to-wall,  curb-to-curb 

22 

Maximum  grade  capability  -  longitudinal  and  vertical 

23 

Portability  to  real-time  simulator 

24 

Error  Handling  /  Diagnostic  Reason  Codes-  Easy  to  troubleshoot 

25 

Multi-pass  vs.  Single  Pass  results 

26 

Average  and  Minimum  RCI  values 

27 

Rut  depths  with  spatial  location  data 

Table  11-5:  Necessary  Next-Gen  NRMM  Output  Products  /  Considerations 
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Topic 

Considerations 

Data  Availability 

How  easy  /  difficult  will  it  be  to  obtain  this  data  for  blue  /  red  systems? 

How  much  data  is  required  to  have  sufficient  confidence  in  the  simulation  results? 

Even  if  we  could  obtain  the  data,  will  sufficient  time  be  allotted  to  develop  a  model  given  the  analysis  deadline  constraints? 

Does  the  tool  /  model  need  to  possess  a  terrain  library  -  multiple  locations,  selectable  pre-defined  features  (e.g.  standard  obstacles), 
etc? 

Data  Resolution  / 

Data  Scale 

What  resolution  of  data  is  needed  /  can  be  obtained  -  e.g.  terrain,  remotely  sensed  terrain,  vehicle  data,  etc. 

What  level  of  vehicle  data  can  be  obtained  given  the  maturity  of  the  design  (e.g.  paper  concept,  technology  demonstrator,  fielded 
system,  etc.) 

How  do  we  handle  /  aggregate  uncertainty  associated  with  model  input  data? 

Given  the  scale  of  the  system  under  evaluation  (e.g.  full-scale  vehicle  vs.  man-portable  robot)  how  are  terrain  features  handled  (e.g. 
terrain  features  encountered  by  a  vehicle  might  affect  surface  roughness  while  the  man-portable  robot  considers  them  to  be 
obstacles)? 

Can  the  tool  /  tools  accommodate  multiple  levels  of  data  fidelity  (e.g.  from  detailed  designs  with  high  levels  of  data  to  paper  concepts 
with  high  levels  of  data  aggregation  /  surrogation  at  the  subsystem  level)?  For  example,  the  system  could  accommodate  reduced  order 
modeling  (if  there  was  if  insufficient  time  /  data)  or  run  higher  fidelity  component  level  performance  modeling  based  on  used  selected 
criteria. 

Level  of  Customization 

How  difficult  /  easy  will  it  be  to  customize  the  software  to  accommodate  non-traditional  mobility  systems  (e.g.  towed,  multi-unit 
systems,  bi-pedal  robots,  driven  wheel  hubs,  new  control  systems,  etc.)? 

Are  there  specific  technologies  unique  to  tracked  vs.  wheeled  vehicles?  Do  these  pose  any  specific  modeling  challenges? 

What  scripting  languages  and/or  tools  and  modules  can  be  integrated  to  add  additional  capabilities?  (i.e.  APIs) 

Stochastic  vs. 

Deterministic 

Should  the  model  provide  results  that  achieve  one  and  /  or  both? 

Modularity  /  Future 
Growth 

Does  the  model  need  to  be  able  to  support  future  additional  capabilities  through  use  of  "add-ons",  "plug-ins",  "logic  modules",  etc? 

Can  the  model  support  the  incorporation  of  new  /  novel  technologies? 

Open  Source /GOTS  vs. 
Proprietary  Output 

Can  the  results  be  exported  to  other  analysis  /  visualization  tools  or  all  the  results  proprietary  to  the  tool? 

Ease  of  Use 

Ability  to  reuse  data  models,  automate  runs,  capability  to  complete  "batch"  runs,  etc. 

Can  the  model's  inputs  be  databased  and  queried  to  quickly  identify  existing  vehicle  models  (for  use  in  subsequent  studies)? 

Steady  State  vs.  Non 
Steady-State 

Must  the  system  be  able  to  handle  non-steady  state  inputs  (e.g.  driver  acceleration)? 

Real-time  vs. 

Non-real-time 

Must  the  results  be  generated  in  real-time  or  a  near-real-time  condition  (e.g.  driving  simulation  /  human-in-the-loop)  or  can  the  results 
be  generated  over  a  longer  duration  timeframe? 

Data  Standards 

Are  there  specific  data  input  /  output  standards  that  are  required  (e.g.  Open  Geospatial  Consortium  (OGC)  or  ISO  2631  /  ISO  8608  -  ride 
quality)? 

Are  there  specific  software  program  requirements  (e.g.  Matlab,  ArcGIS,  etc)? 

What  are  the  data  interoperability  standards  needed  to  import  and  export  data  into  the  model  (i.e.  FMI  -  Functional  Mock-up  Interface, 
Geospatial,  etc) 

Ability  to  incorporate 
spatially-oriented  data 

Can  the  tool  /  tools  incorporate  GIS  data?  If  so,  what  formats?  Do  the  model  outputs  contain  a  spatially-oriented  data  that  can  be 
visualized  in  GIS  visualization  products? 

GUI  based  or  Command 

Line 

Do(es)  the  tool(s)  possess  an  easy  to  use  graphical  user  interface  or  does  the  user  need  to  execute  the  model  from  the  command  line? 

2D  vs  3D 

Two  dimensional  vs.  three  dimensional  analysis  capability  required? 

Urban  Maneuverability 

Ability  to  model  mobility  challenges  associated  with  maneuvering  in  urban  and/or  other  similarly  constrained  environments. 

Modes  of  Operation 

Novice,  Intermediate,  Advanced  User  Modes  -  to  enable  users  of  all  experience  /  proficiency  levels  to  utilize  the  tool 

Are  "role"  based  user  interfaces  a  more  suitable  approach  (i.e.  Developer,  Practitioner,  Supervisor  /  Practitioner,  Novice  /  Operational 
User)? 

Can  the  model  incorporate  Hardware-in-the-Loop? 

Can  the  model  support  batch  runs? 

Table  11 

1-6:  Other  Data  Input  /  Output  Considerations  for  the  Next-Gen  NRMM 
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1 1.4  INPUT  DATA  /  OUTPUT  POTENTIAL  NEAR-TERM  STOP-GAP 
SOLUTIONS 

At  the  NATO  meetings  in  Brussels,  Belgium,  AMSAA  presented  some  potential  solutions  they  developed  to 
address  short-term  NRMM  capahility  gaps.  Three  products  were  highlighted: 

1.  The  System  Level  Analysis  Mobility  Dashboard  (SLAMD)  -  a  Python-based  NRMM  wrapper  that 
improves  the  end-user  experience,  integrates  the  various  NRMM  modules  (ObsMod,  VehDyn,  etc.) 
into  one  user  interface,  reduces  vehicle  file  development  time  with  improved  error  handling 
capabilities,  improves  data  post-processing  capabilities,  etc. 

2.  The  AMSAA  Urban  Maneuverability  Model  (UMM)  -  a  custom-built  ESRI  ArcGIS  /  Python  tool  that 
can  be  used  to  address  vehicle  urban  maneuverability  analysis  capability  gaps 

3.  The  AMSAA  Optimal  Path  Model  (AOPM)  -  a  custom-built  ESRI  ArcGIS  tool  that  incorporates 
NRMM  on-road  and  off-road  speed  and  trafficability  predictions  to  plot  the  optimal  path  between 
geospatially-oriented  point  locations 

11.4.1  System  Level  Analysis  Mobility  Dashboard  (SLAMD) 

AMSAA  has  realized  the  following  benefits  since  developing  SLAMD: 

1 .  Improved  consistency  in  analysis  methodology  across  all  NRMM  users 

2.  Streamlined  analysis  processes  to  allow  users  to  more  quickly  respond  to  customer  requests, 
including  vehicle  configuration  changes,  support  trades  analyses 

3.  Automation  of  repetitive  data  collection  and  post  processing  tasks  to  permit  more  time  for  in- 
depth  analysis  of  results 

4.  Leveraged  existing  analysis  tools  (NRMM,  VEHDYN,  etc.)  without  re -coding  them 

5.  Databased  model  inputs  and  outputs  to  improve  analysis  efficiency 

6.  Includes  elements  to  streamline  use  of  NRMM  and  other  potential  M&S  tools 

7.  Configuration  management  and  control  of  all  Input  /  Outputs  data  elements  through  the  use  of 
a  centralized  data  storage  repository. 

Eigure  11-1  below  shows  the  current  text -based,  command  line  NRMM  Input  data  files  as  compared  to  the 
improved  GUI  interface,  data  development  environment  provided  by  SLAMD  (shown  in  Eigures  11-2  to  11- 
4). 
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’  sample, veh  -  Notepad 
File  Edit  Format  View  Help 

vehi cle 
VD4 . 2  Text 
22 

0.00,54560 
2.81.47974 
3.01,40204 
2.64,39073 
2.89,38875 
3.59,37496 
4.07,33912 
5.19,29642 
5.40,26464 
6.70,20297 
8.02,18071 
8.71,16845 
10.8,15663 
13.1,13256 
15.9,10488 
18.1,8789 
22. 5,7533 
27. 5,6122 
32.6,5817 
38.1,4625 
42.8,3571 
44.0,828 

I, 1,903,8,10. 5,1265 
0.75,97,0.5,0.8,9999.9 
1,2, 2, 0,0,0 
12,0,0,0,16 

-1,0,1,2,4,6,12,15,17,19.14. 
-88888,0,100,200,400,500,906,1100,1200,1550,3050,4013 

II, 0,0,0,22.14 

-1,0,2,4,6,12,15,17,19.14,22.14,23.14 

-65987,0,600,1100,2300,5400,8200,14500,18530,25672,44444 
7  n  n 


u.  10 
0.27 
0.04 
0.06 
0.07 
0.02 
0.10 
0.04 
0.04 
0.08 
0.03 
0.09 
0.25 
0.11 
0.03 
0.04 
0.02 
0.08 
0.07 
0.04 
0.09 
0.15 
0.08 
0.19 
0.09 
0.06 
0.05 
0.06 
0.02 
0.05 
0.16 
0.03 
0.05 
0.01 
0.03 
0.08 


u 

46.0 

6.0 

2.0 

4.0 

30.0 

25.0 

29.0 

33.0 

7.0 

6.0 

49.0 

2.0 

19.0 

4.0 

4.0 

5.0 

15.0 

23.0 

23.0 

9.0 

8.0 

83.0 

10.0 

30.0 

49.0 

60.0 

35.0 

39.0 

49.0 

24.0 

2.0 

3.0 

76.0 

22.0 

28.0 


1U.4/ 
0.00 
9.  38 
6.  31 
9.73 
10.  38 
5.82 
5.20 
0.00 
0.00 
0.00 
0.00 
40.24 
0.00 
4.00 
12.  54 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
6.22 
14.  32 
18.61 
0.00 
0.00 
7.90 


/  .  34 
0.00 
8.  57 
6.  31 
9.25 
6.10 
4.10 
3.78 
0.00 
0.00 
0.00 
0.00 
35.01 
0.00 
4.00 
12.43 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
2.69 
0.00 
0.00 
0.00 
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0.00 
0.00 
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6. 31  8 

9.87  8 

16.00  1 
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0.00  -5 
0.00  -7 
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0.00  -7 
4.00  9 
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4.00  9 

12.60  1 
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Figure  11-1:  Existing  Text-Based  /  Command  Line  Interfaces  for  NRMM  Input  Data  File 

Construction  and  Execution 


Figure  11-2:  SLAMD  Improved  Vehicle  Data  Creation  Interface  (Template-Based) 
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Check  value  in  field:  M1080_2.Primary_Mover. Suspension!. NI. II. LWHL! 
(M  1080_2.Primary_Mover. Suspension  l.NI. II. LWHL) 


CG-Wheel  Center  On)*(LWHL) 
min  ground  clearance  for  each 

OMTKI 


MM  calc  (mph)  (Highway)-{VTTRMX_Hwy) 

MM  calc  (mph)  (Cross  Country)-(mRMX_XC) 

MM  calc  (mph)  (Mud  /Sand/Snow)-(VnRMX_MSS) 
MM  calc  (mph)  (Emergency)*(VTIRMX_Emr) 

/elfbr  tracked  w/trailing  arm  Cn)-0iROSUS_l) 


60 

IF? 


liS 


Wheel! 


12010 

None 

None 

13.75 

None 

«.3  1 

None 

Tire^oad  Wheel  Diameter  On)-(DIAW) 

Figure  11-3:  SLAMD  Improved  Data  Validation  /  Error  Handling 


Figure  11-4:  SLAMD  Graphical  User  Interface  (GUI)  for  VehDyn,  ObsMod,  NRMM 
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SLAMD’s  Graphical  User  Interface  (GUI)  steeply  reduces  the  learning  curve  associated  with  learning  how  to 
use  the  NRMM.  This  improved  user  interface  delivers  the  following  benefits: 

•  Steeply  reduces  NRMM  learning  curve  and  makes  it  accessible  to  all  user  experience  levels 

o  Eliminates  need  to  learn  NRMM  variable  names  and  parameters 

o  Transitions  NRMM  from  command  line  execution  to  GUI-based  execution  which  is  more 
intuitive  to  users 

o  Provides  “help”  functions  through  the  GUI  to  assist  users  with  data  input  to  support  vehicle 
file  creation 

o  Incorporates  data  validation  -  to  ensure  input  data  results  are  reasonable  and  “flags”  values 
that  are  beyond  reasonable  ranges  for  further  user  investigation 
o  Consolidates  all  NRMM  executables  into  one  easy-to-use  interface 

•  Facilitates  improved  post-process  visualization  of  multiple  vehicle  /  scenario  NRMM  results 

SLAMD  (or  another  similar  approach)  might  be  able  to  address  some  of  the  use  /  usability  capability  gaps 
until  the  release  of  the  Next-Generation  NRMM. 

11.4.2  AMSAA  Urban  Mobility  Model. 

AMSAA  had  previously  been  working  to  address  another  capability  gap  identified  by  the  NATO- 
AVT  ET-148  membership  -  urban  maneuverability  modeling. 
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Evaluating  Add-on  Armor  Technologies 


Pie-shaped  “Wedges”  - 
Indicate  Turning  Results 


•  Graphic  depicts  Go  /  NoGo  roads  and  turning  combinations  for  the  exampie  vehicle  with  add-on  annor  applied 

•  AMSAA  modeling  methodology  evaluates  the  vehicle’s  physical  dimensions  and  turning  capability  on  its  performance  in 
operational  environments 

■  Product  delivered  to  operational  commander  to  inform  mission  planning 


77  sq.  km 

9,229  roads  attributed 
1 1 , 1 02  turning  opportunities 


m 


Notional  Data 


Spatial  Visualization  of  results 


Figure  11-5:  Notional  Urban  Maneuverability  Analysis  Product  -  Evaluating  Maneuverability 

Degradation  Associated  with  Add-On  Armor 

AMSAA’ s  Urban  Maneuverability  Model  (UMM)  leverages  high -resolution  satellite  imagery  and  vehicle 
performance  and  characteristics  data  to  analyze  vehicle  maneuverability  performance  on-road  within 
constrained  urban  environments.  Geospatial  software  is  used  to  calculate  the  geometry  of  the  road  networks 
and  overlay  vehicle  performance  to  create  cartographic  products. 

The  model  requires  a  road  network  to  be  digitized  using  high  resolution  satellite  imagery  and  the  features 
attributed.  Digitization  is  the  extraction  of  features  such  as  road  networks,  canals,  bodies  of  water,  buildings, 
etc,  and  it  also  establishes  the  geospatial  location  of  the  object.  When  a  road  is  digitized  it  is  represented 
spatially  by  a  series  of  polylines  which  connect  to  form  the  road  network.  A  polyline  is  a  feature  that  consists 
of  line  segments  connected  to  each  other  to  form  a  line. 

As  these  polylines  are  created  they  are  saved  to  a  shapefile,  which  is  a  file  that  consists  of  geospatial  vector 
data.  Vector  data  can  include  points,  lines,  and  polygons,  and  it  is  the  backbone  of  most  geospatial  analysis. 
The  attribution  process  involves  associating  important  feature  properties  /  characteristics  to  each  geospatial 
feature.  The  software  allows  the  model  to  extract  road  network  information  such  as  road  width,  road 
construction,  number  of  lanes,  etc. 

The  extracted  features  are  overlaid  onto  a  terrain  area  to  verify  all  features  have  been  properly  extracted  from 
high  resolution  satellite  imagery.  By  overlaying  features,  the  geospatial  software  is  able  to  provide  a  multi¬ 
dimensional  view  of  the  various  data  layers  and  combine  information  between  feature  layers.  This  process 
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extends  the  analysis  capability  by  adding  information  to  the  road  attributes  for  slope,  soil  type,  and  moisture 
content  that  is  not  inherent  in  the  road  layer  alone.  Once  the  road  network  has  been  digitized  and  attributed,  a 
vehicle’s  maneuverability  performance  can  be  analyzed.  Statistical  and  cartographic  products  can  be  created 
to  quantify  and  visualize  the  results.  In  the  graphic  above,  the  color-coded  roads  indicate  whether  or  not  a 
vehicle  can  “fit”  down  the  road,  while  the  color-coded  pie-shaped  wedges  between  the  roads  indicate  whether 
or  not  a  vehicle  can  negotiate  a  turn  from  one  road  to  another  (Green  =  Go  /  Red  =  NoGo).  AMSAA  has 
further  refined  the  model  to  evaluate  the  connectivity  of  the  road  network  -  essentially  removing  any  areas 
deemed  “Go”  but  offer  no  viable  path  into  /  out  of  this  area  of  the  road  network. 

AMSAA  has  historically  run  this  model  to  inform  vehicle  design  decisions  regarding:  the  physical  dimensions 
of  vehicles;  modifications  to  the  steering,  driveline,  and  suspension  systems  (which  may  affect  the  turning 
capability  of  the  vehicle);  and  the  effects  of  add-on  armor  technologies. 

The  modular  nature  of  the  NRMM  terrain  files  and  the  ability  to  import  /  export  spatially -oriented  GIS  terrain 
data  enables  NRMM  results  to  be  visualized  cartographically.  Despite  the  complexity  of  the  various  terrain 
input  data  layers  (i.e.  slopes,  soils,  moisture  content,  surface  roughness,  etc),  GIS  software  enable  users  to 
spatially  join  these  layers  together  to  create  new  NRMM  terrain  files.  Figure  11-6  below  depicts  a  notional 
comparative  speed  /  trafficability  analysis  of  two  vehicles  operating  in  Lauterbach,  Germany  with  a  snow 
scenario. 


Visualization  and  analysis  of  multiple  vehicle  outputs  using  ArcGIS  (COTS)  softvt/are. 


Notional  Data 


Figure  11-6:  Notional  Vehicle  Speed  /  Trafficability  Comparison  Product  Generated  Using 

NRMM  and  ESRI  ArcGIS 


AMSAA  has  historically  exported  the  statistical  results  of  NRMM  into  GIS  software  for  additional  analysis  / 
visualization  purposes.  At  the  NATO  AVT  ET-148  meeting  in  Brussels,  attendees  confirmed  that  both  the 
French  and  German  militaries  were  developing  similar  geospatial  mobility  analysis  capabilities;  however. 
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since  some  of  these  activities  were  tied  to  mission  /  operational  planning  capabilities,  they  were  elassified  at 
the  NATO//SECRET  level  or  above. 

11.4,3  AMSAA’s  Optimal  Path  Model 

AMSAA’s  Optimal  Path  Model  (AOPM)  enhances  the  potential  spatial  analysis  capabilities,  inherent  within 
NRMM’s  modular  terrain  data  framework,  by  enabling  the  importation  of  NRMM  on-road  and  off-road  speed 
and  trafficability  predictions;  and,  plotting  the  optimal  path  between  geospatially-oriented  point  locations. 

NRMM’s  modular  terrain  framework  allows  end  users  to  import  GIS  terrain  data  into  spatial  analysis  tools 
such  as  ESRI’s  ArcGIS.  Then,  ArcGIS  can  be  used  to  generate  new  NRMM  terrain  units  that  represent  each 
unique  combination  of  the  terrain  characteristics  present  within  the  terrain  playbox.  NRMM  can  ingest  the 
new  terrain  file,  built  with  these  new  NRMM  terrain  units,  to  make  on-road  and  off-road  speed  predietions. 
AMSAA’s  0PM  can  then  import  the  NRMM  results  and  aggregate  the  on-road  and  off-road  performance  into 
a  single  speed  performance  map.  Additional  “cost  surfaces”  can  be  added  to  incorporate  other  path  modeling 
eonsiderations,  (i.e.  fuel  economy,  concealed  movement,  enemy  engagement  ranges).  The  model  then  uses 
Dijkstra’s  algorithm  to  optimize  the  path  across  the  combined  cost  surfaces  to  find  the  optimal,  idealized  path 
through  the  network  of  points.  Eigure  11-7  below  provides  a  flow  chart  outlining  the  steps  in  the  AOPM 
methodology.  The  model  enables  mobility  performance  results  to  be  evaluated  within  specific  mission 
contexts  as  shown  in  the  Mission  Completion  Time  Estimates  generated  for  vehieles  eondueting  Medieal 
Evacuation  (MEDEVAC)  missions  -  see  Eigure  1 1-8. 


* 


Path  Distance 
Accumulated  Least 
TravelTimc  Map 


Figure  11-7:  Notional  MEDEVAC  Mission  Effectiveness  Product  Generated  Using  NRMM  and 

ESRI  ArcGIS 
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Path  modeling  results  applied  to  mission  Medical  Evacuation 
(MEDEVAC)  scenario 


AMSAA 


ImvMvm*  la  AMlfw* 


Wilh  selected  POI  /  CCP  and 
FOB  locations  in  rolling  terrain  of 
Germany 

«■  All  ME  alternatives  achieve 
the  mission  time  requirement 
(within  60  mvi.)from  POI  / 
CCP  to  FOB 

Best  performing  vehicle  ts  ME 
vehicle  2  -  achieves  ME 
mission  vwth  the  least  travel 
lime  in  all  cases 


■PCNS/CCP) 

■PCM2;CCP2 

■poiiyccpi 


Figure  11-8:  Notional  MEDEVAC  Mission  Effectiveness  Product  Generated  Using  NRMM  and 

ESRI  ArcGIS 


Therefore,  the  Next-Generation  NRMM  must  retain  the  capability  to  import  geospatial  terrain  data  and 
comply  with  military,  commercial,  and  Open  Geospatial  Consortium  (OGC)  data  standards  to  preserve  data 
interoperahility  between  analysis  tools.  Additionally,  the  results  generated  by  the  Next -Generation  NRMM 
should  be  able  to  be  exported  and  visualized  using  CIS  analysis  and  cartographic  visualization  software. 


11.5  FUTURE  WORK  /  RECOMMENDATIONS 

Near  Term: 

•  Continue  to  develop  modular  interim  solutions  to  close  vehicle  /  terrain  modeling  gaps  and  /  or 
address  end  user  usability  issues. 

•  Improve  methodologies  to  transform  high  resolution  satellite  imagery  /  remotely -sensed  CIS  data  into 
accurate  NRMM  terrain  representations. 

•  Investigate  the  potential  to  develop  data  /  interface  standards  to  promote  data  interoperability  between 
Multi-Body  Vehicle  Dynamic  simulations  and  commercial  GIS  software  solutions. 

•  Map  the  Input  Data  Requirements  /  Output  Products  to  end  user  roles  /  user  experience  levels. 

•  Map  the  Input  Data  Requirements  /  Output  Products  to  various  modeling  levels  (Reduced  Order 
Modeling  through  Detailed  Engineering  Analysis). 
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Long  Term: 

•  Pursue  a  modular  development  approach  -  leveraging  Vehicle  Multi-Body  Dynamic  Analysis  Tools, 
Geospatial  Terrain  Development  /  Cartographic  Visualization  Tools. 

•  Publish  Next-Generation  NRMM  Data  Interoperability  Standards  -  to  ensure  NRMM  outputs 
maintain  linkages  to  spatially  oriented  data  to  facilitate  visualization  using  COTS  CIS  tools. 

•  Pursue  Scalable  Levels  of  Fidelity  to  Model  Systems  from  Paper  Concepts  to  Detailed  Engineering 
Designs  (accommodating  expedient  to  more  lengthy  analysis  timelines). 

•  Incorporate  modules  to  model  many  of  the  advanced  vehicle  technologies  identified. 

•  Incorporate  improvements  to  the  terrain  /  environment  development  processes;  and  the  operator 
behavior,  human  factors,  and  autonomous  /  semi-autonomous  vehicle  characterization  methodologies. 
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Chapter  12  -  THEME  7:  VERIFICATION  &  VALIDATION 


Michael  Letherwood 


12.1  GOALS  AND  DELIVERABLES 

The  goals  of  Theme  7  are  to  provide  a  process  for  conducting  a  successful  tool  Verification  and 
Validation  (V&V)  program  on  the  Next  Generation  NRMM  (NG-NRMM).  The  intent  of  the  ET  is  the 
development  of  a  set  of  standards  to  guide  the  implementation  of  the  NG-NRMM,  as  well  as  its  use  and 
management.  It’s  driven  by  the  need  for  highly  accurate  numerical  models  for  making  vehicle  system 
mobility  and  performance  capability  predictions  to  support  current  systems  as  well  as  future 
acquisition  programs.  The  expected  deliverable  of  Theme  7  is  a  benchmarking  verification  and 
validation  plan  to  assess  potential  NG-NRMM  developers’  modeling  methodologies,  capabilities, 
and  component  models  for  vehicle  dynamics,  off-road  mobility,  intelligent  vehicle  operation,  and 
geospatial  data  use  and  mapping,  which  will  ultimately  lead  to  the  development  of  a  set  of  standards 
to  guide  the  implementation  of  NG-NRMM,  as  well  as  its  use  and  management.  Software  V&V  is 
fundamentally  different  from  model  V&V  and  is  required  when  a  computer  program  or  code  is  the 
end  product  and,  conversely,  tool  V&V  is  required  when  a  predictive  model  is  the  end  product.  As 
such,  this  report  will  discuss  primarily  tool  V&V  activities  and  evaluation  of  developer’s  responses 
to  see  which  groups  can  adequately  address  the  long  list  of  NG-NRMM  requirements. 

The  Theme  7  path  forward  deliverables  are  to: 

•  Phase  I:  To  conduct  a  Tool  Benchmarking  V&V  with  developers  to  provide  a  common  basis 
for  evaluating  tool  capabilities  in  the  context  of  NG-NRMM  requirements 

•  Phase  II:  To  develop  NG-NRMM  standards  version  1.0  and  associated  benchmarks  and  to 
establish  the  basis  and  process  for  on-going  future  development,  configuration  management, 
and  tool  qualification 


The  theme  members  are  shown  below: 


Countries 

Name 

Denmark 

Balling,  Ole 

Germany 

Gericke,  Rainer 

USA 

Gunter,  David 

USA 

Jayakumar,  Paramsothy 

USA 

Letherwood,  Michael:  Leader 

USA 

McCullough,  Michael 
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12.2  OBJECTIVES 

The  ET’s  Theme  5,  Tool  Choices  team  was  able  to  effectively  identify  critical  elements  of  a  physics- 
based,  next  generation  mobility  model  utilizing  strength  and  weakness  criteria  provided  by  an  initial 
“pros  and  cons”  review  of  the  current  NRMM  and,  subsequently,  integrate/coordinate  those  tool 
choice  evaluations  with  other  themes,  particularly  requirements  and  methodology  themes.  They  went 
on  to  identify  potential  solutions  throughout  the  technical  community  and  user  nations  and  then 
surveyed  the  ability  of  current  and  future  physics -based  simulation  environments  to  provide  accurate 
and  timely  results  that  can  be  used  to  support  vehicle  system  development,  acquisition,  prediction  of 
vehicle  performance  in  an  adverse  operational  environment,  and  force  projection  metrics.  They  were 
able  to  investigate  the  ability  of  a  limited  number  of  commercially  available  physics -based 
simulation  tools  to  address  the  needs  of  the  current  NRMM  tool  set  and  determine  the  ability  of  those 
tools  to  augment  empirically  based  historic  analytical  solutions  providing  a  path  to  full  physics-based 
analysis  and  prediction  of  the  vehicle-terrain  interaction.  The  team  successfully  completed  those 
taskings  and  the  job  of  developing  a  plan  to  evaluate  those  capabilities  fell  to  the  Theme  7  Team: 
Verification  &  Validation.  Although  late  getting  started,  the  objectives  of  the  team  has  been, 
ultimately,  to  verify  &  validate  NG-NRMM  prospective  objective  methodologies  of  component 
models  for  off-road  mobility,  vehicle  dynamics,  and  intelligent  vehicles. 

Hence,  the  Phase  I,  Tool  Benchmarking  V&V  with  developers  is  intended  to  provide  a  common 
basis  for  evaluating  tool  capabilities  in  the  context  of  NG-NRMM  requirements.  The  objectives  are 
to; 

•  Determine  if  adequate  physics  based  M&S  tools  exist  either  in  the  public  domain  or  can  be 
provided  by  industry 

•  Determine  if  those  tools  can  be  used  to  accurately  represent  the  key  mobility  elements  which 
affect  ground  vehicles 

•  Determine  if  those  tools  are  affordable  and  implementable 


The  benefits  for  prospective  software  developers  will  be  to: 

•  Gain  familiarity  with  the  development  of  NG-NRMM  program  requirements 

•  Provide  current  data  which  can  be  used  to  inform  the  requirements 

•  Demonstrate  the  realm  of  the  possible 

•  Recognize  the  simulation  capability  gaps 

•  Provide  off-the-shelf  simulation  tools  to  relevant  NATO  nations  and  vehicle  OEMs 

•  Improve  capabilities  utilizing  the  NATO  benchmark 

•  Suggest  additional  applicable  benchmarks 

12.3  QUESTIONS  TO  BE  ADDRESSED 

As  discussed  earlier,  since  the  final  NG-NRMM  standards/code  is  still  a  work  in  progress,  the  NATO 
RTO  Task  Group  committee  will  define  the  full  scope  of  the  resulting  Phase  II  NG-NRMM  Code 
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V&V  efforts.  The  Phase  I,  Tool  Model  Benchmarking  V&V  discussions  resulted  in  the  following 
open  questions  that  were  posed  and  addressed  as  follows; 

1 .  What  problems  or  events  or  scenarios  do  we  need  to  V&V? 

The  following  events  will  be  used  during  benchmarking  exercise: 

1 .  Steady  State  Cornering 

2.  Double  Lane  Change  w/wo  Autonomy 

3.  Side  Slope  Stability 

4.  Grade  climbing 

5.  Ride  and  Shock  Quality 

6.  Step  climb  and  ditch  crossing 

7.  Off  road  trafficability  w/wo  autonomy 

8.  Urban  navigation  at  different  levels  of  autonomy 

2.  What  vehicles  do  we  want  to  use  for  the  benchmarking? 

•  Wheeled  Vehicle 

•  Tracked  Vehicle 

3.  What  test  data  are  available  and  who  can  provide  the  test  data? 

Wheeled  Vehicle 

•  TBD 

Tracked  Vehicle 

•  Drawbar  pull  force  vs.  slip  -  on  sandy  terrain  (LETE  Sand),  muskeg  (Petawawa  Muskeg  B), 
and  snow  (Petawawa  Snow  A) 

•  Bevameter  parameters  -  for  sandy  terrain  (EETE  Sand),  muskeg  (Petawawa  Muskeg  A  and 
B),  and  snow  (Petawawa  Snow  A  and  B) 

4.  What  vendor  tools  do  we  want  to  benchmark  against  the  test  data? 

Based  on  the  results  of  the  Theme  5:  Tool  Choices  team  Request  for  Information  (REI),  the  top  eight 
best-qualified,  prospective  developers  were  selected  to  visit  the  ET-148  committee  during  the  NATO 
meeting  in  Prague  and  to  describe  their  capabilities.  One  of  the  questions  that  will  need  to  be 
answered  is  whether  to  re-engage  only  the  original  developers  or  to  invite  others  to  participate.  It  is 
expected  that  the  technology  associated  with  prediction  of  vehicle  performance  in  extreme  conditions 
will  continue  to  improve  and  therefore  new  tools  may  be  available  throughout  the  process.  As  the 
efforts  move  forward  the  ET  and  RTG  committees  will  continue  to  share  lessons  learned  and  will  use 
that  information  to  establish  suitable  benchmarks,  dominant  criteria,  integration  of  terrain  and 
vehicle  parameters  etc 
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5.  Will  any  additional  tests  need  to  be  done  during  the  benchmarking  exercise? 

At  this  time  it  has  not  been  decided  what  new  tests  need  to  be  run  to  support  the  benchmarking 
exercise.  Rainer  Gericke  is  prepared  to  collect  more  MAN  truck  data  if  necessary. 


12.4  TEST  VEHICLES 

A  description  of  the  two  test  vehicles  is  detailed  below. 
Wheeled  Vehicle 


•  TBD 
Tracked  Vehicle 


•  Fully  tracked  armored  personnel  carrier 

•  Detroit  6V53  V6  two-stroke  diesel  engine  of  318  cubic  inches  (5,210  cc)  with  an  Allison  TX- 
100-1  3-speed  automatic  trans 

•  Aluminum  armor  that  made  the  vehicle  much  lighter  than  earlier  vehicles  and  very  mobile 

•  Vehicle  total  weight,  sprung  and  unsprung  weight 

•  Sprung  weight  x  (long.)  and  y  (vert.)  CG  coordinates 

•  Drawbar  hitch  x-coordinate  and  y-coordinate 

•  Fixed  (  sprocket/tensioning)  wheels  -  wheel  radius,  x  and  y  coordinates  of  wheel  centers 

•  Torsion  Bar  Suspension/Road  Wheels  -  x  and  y  coordinates  of  pivot  points,  arm  angles  at  free 
positions  (i.e.,  the  angular  positions  of  the  arms  at  which  suspension  spring  elements  are  not 
subject  to  any  load),  torsion  bar  stiffness,  wheel  radius 

•  Track  parameters  -  weight  per  unit  length,  width,  pitch,  grouser  height,  thickness,  track  tension- 
elongation  relationship 

•  Initial  track  tension  at  rest 

•  Static  equilibrium  position,  wheel  loads,  and  natural  frequency 

•  Belly  shape 

•  Wheel  centers 

•  Drawings  -  in  3  dimensions  showing  locations  (attachment  points)  of  the  chassis,  major 
component  eg  locations,  vehicle  hitch  point;  suspension  system  components  trailing  arms,  torsion 
bars,  panhard  bars,  torque  rods,  chains,  etc. 


12.5  SOETWARE  DEVELOPERS 

Based  on  the  results  of  the  RFI  developed  by  Theme  5,  the  top  eight  software  developers  were 
invited  to  Prague  to  present  their  capabilities.  A  brief  summary  is  below.  All  were  invited  to 
participate  in  the  Benchmarking  exercise. 
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Advanced  Science  &  Automation  (ASA):  Tamer  Wasfy  described  the  software  package  known  as 
IVRESS/DIS.  DIS  stands  for  Dynamic  Interactions  Simulator.  It  incorporates  multi -body  dynamics 
(MBD),  Finite  Element  Models  (FEM),  Discrete  Element  Models  (DEM),  and  Smoothed  Particle 
Hydrodynamics  (SPH)  with  pre-processors  for  user-friendly  or  expert  applications. 

CM  Eabs:  Justin  Webber  and  Sebastien  Miglio  discussed  their  Vortex  Dynamics  software,  which 
was  spun  off  from  MathEngine.  They  stressed  Vortex  as  real-time  simulation  software.  Their 
expertise  is  in  autonomous  driving  and  driver- in-the-loop  simulations.  They  use  real  vehicles  to 
create  simulation  training.  Vortex  is  not  FEA,  but  a  Simulation  Development  Platform.  It  can  run 
real-time  simulations  on  an  ordinary  PC. 

Dassault  Systems  3DS:  Bob  Solomon  and  Frederic  Dot  represented  Dassault  and  described  their 
Simpack  software,  which  was  recently  purchased  by  Dassault.  Simpack  technology  was  developed 
by  DER,  the  German  aerospace  group.  They  can  do  co-simulation  with  Abaqus  EE  application, 
which  provides  a  powerful  soil  model.  They  don’t  currently  do  track  simulations. 

FunctionBay:  Uwe  Eiselt  presented  the  information  about  their  MBD  software  known  as  Recurdyn. 
The  work  started  in  South  Korea  in  the  1990s.  They  have  a  fully  integrated  EE  model.  They  include 
DEM  through  a  third  party,  but  it  is  also  integrated  in  their  software.  They  showed  some  simulations 
demonstrating  autonomous  control.  They  produce  both  an  easy  Excel  version  for  less  skilled  users 
and  ProcessNet  for  skilled  programmers.  They  stress  ease  of  use. 

Modelon:  Hubertus  Tummescheit  presented  the  material  from  Modelon,  which  began  in  Sweden.  He 
emphasized  that  you  should  tie  yourself  to  standards,  not  to  tools.  He  discussed  the  software  tool, 
Dymola,  for  simulating  the  dynamic  behavior  of  systems.  It  is  based  on  the  Modelica  open  standard 
for  component-oriented  modeling  of  complex  systems  and  includes  the  Functional  Mock-Up 
Interface  (EMI)  toolbox.  Modelica  was  selected  by  DARPA  for  their  FANG  challenge.  Due  to  the 
open  code,  the  user  can  drill  down  and  find  the  relevant  equations  and  change  them  if  needed. 
Dymola  can  produce  real-time  simulations.  They  believe  that  to  do  autonomy,  you  must  have  real¬ 
time  simulations  or  know  the  latency  exactly.  They  have  not  done  soft-soil  simulations  or  dealt  with 
tracked  vehicles.  They  would  concentrate  on  a  Chrono  integration  for  tracked  vehicles. 

MSC  Software:  Peter  Dodd,  Kyle  Indermuehle  and  Henrik  Skovbjerg  were  visitors  from  MSC 
Software  and  described  their  Adams  software.  The  firm  started  50  years  ago  as  an  offshoot  from 
NASA.  They  produce  Adams/Car  and  Adams/ATV,  a  toolkit  for  tracked  vehicles  on  soft  soil.  They 
were  unsure  if  they  could  handle  our  soft  soil  applications  and  have  not  dealt  directly  with  intelligent 
vehicles.  They  use  EDEM  Co-Simulation  for  DEM  work,  such  as  for  soft  soil. 

Siemens:  Sebastian  Flock  and  lurie  Terna  discussed  Siemen’s  software,  LMS  Virtual  Lab,  also  with 
EMI  compatibility.  They  also  use  EDEM  for  soft  soil  applications.  They  do  not  have  expertise  in 
geospatial  terrain  or  autonomy  applications.  On  the  positive  sign,  one  of  their  slides  showed  a  quote 
from  Mike  McCullough  touting  their  product.  The  software  can  be  leased  or  purchased. 
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U.  of  Wisconsin:  This  team  included  Dan  Negrut,  Radu  Serban,  Alessandro  Tasora  and  Hiroyuki 
Sugiyami  from  U.  Wisconsin  and  Brian  Gerkey  from  Open  Source  Robotics  Foundation  (OSRF). 
Negrut  diseussed  Chrono  software  and  Gerkey  discussed  Gazebo.  Chrono  is  a  toolkit  for  modeling 
and  visualization  of  wheeled  and  tracked  vehicles.  The  University  has  a  super  computer  funded  by 
US  DoD.  As  Negrut  said,  “Hardware  is  Plentiful,  Software  is  Not.”  Gazebo  provides  the  robotic 
application  using  Robot  Operating  System  (ROS)  and  Gazebo  for  robot  simulation.  They  take  pride 
that  their  software  is  all  open-source.  On  the  soil  issue,  they  have  two  projects  with  ERDC  and 
TARDEC.  They  have  submodules  for  the  driveline,  but  they  are  not  validated.  Chrono  has  been 
validated  against  Adams.  They  believe  that  they  can  deal  with  our  events.  With  Gazebo,  they  can 
deal  with  autonomy. 


12.6  TOOL  BENCHMARKING  V&V  SCOPE 

Phase  I  -  “Tool  Benchmarking  V&V”  with  developers  will  be  conducted  as  follows: 

•  Prospective  developers  will  be  provided  with  sufficient  vehicle  data  to  set  up  high-fidelity, 
physics-based  models  of  one  wheeled  and  one  tracked  vehicle 

•  Prospective  developers  will  be  asked  to  simulate  required  performance  scenarios,  and 
subsequently,  provide  their  simulation  data  to  NATO  RTO  task  group  for  evaluation 

•  NATO  RTO  Task  Group  will  evaluate  accuracy  and  capabilities  of  developer  submissions 
The  developer’s  responses  will  be  assessed  by  the  NATO  RTO  Task  Group  committee  as  follows: 


Assessment  Attribute 

•  Score 

Geospatial  Data  Analysis  and  Mapping 

Terrain  modeling  and  visualization  in  compliance  to  CIS  standards 

Able  to  handle  urban  terrain  data 

Supports  sensor-terrain  interaction  modeling 

Mobility  metrics  mapping  tools 

Computational  Physics  of  Vehicle  Terrain  Interaction 

Any  vehicle  morphology 

Pull  range  of  ground  vehicle  geometric  scales 

VTI  models  at  multiple  levels  of  theoretical  and  numerical  resolution  -On  road 

wheels 
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•  VTI  models  at  multiple  levels  of  theoretical  and  numerical  resolution  -On  road 
tracks 

•  VTI  models  at  multiple  levels  of  theoretical  and  numerical  resolution  -Off  road 
wheels  (Bekker-Wong,  etc) 

•  VTI  models  at  multiple  levels  of  theoretical  and  numerical  resolution  -Off  road 
tracks  (Bekker-Wong,  etc.) 

•  Full  coupling  capability  with  FEM/DEM/DVI/SPH  deformable  soil  models 

•  Full  coupling  with  power  trains 

•  Full  coupling  with  embedded  control  systems 

•  Full  coupling  with  flexible  bodies 

•  Amphibious  operations  modeling 

•  Coupling  with  autonomous  and  human  cognition  models 

•  Useful  for  vehicle  design 

•  M&S  environment 

•  Score 

•  Interfaces  to  broad  range  of  tools 

•  Tools  for  automation  and  standardization 

•  Parallelization  and  HPC  compatibility 

•  Tools  for  handling  stochastic  parameters 

•  Modular  interoperability  (ability  to  plug  and  play  subsystems) 

•  Portable  to  most  common  computing  environments 

•  Distributable  to  NATO  designated  stake  holders 

•  Enduring  and  supported  (not  likely  to  become  easily  obsolete) 

•  Expansion  (no  financial,  legal,  technical,  or  architectural  limits  to  mobility  research 
and  development) 

•  Verification  and  Validation  Basis 

•  Verification  and  validation  benchmarks  exist  and  distributable 

•  Verification  basis  is  sound  for  benchmarks  provided 

•  Validation  basis  is  sound  for  benchmarks  provided 

•  V&V  benchmarks  address  NG-NRMM  requirements 
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12.7  SUFFICIENCY  -  VALIDATION  METRICS 


V&V  is  undertaken  to  quantify  confidence  and  build  credibility  in  a  numerical  model  for  the  purpose 
of  making  a  prediction  which  can  be  defined  as  the  “use  of  a  computational  model  to  foretell  the 
state  of  a  physical  system  under  conditions  for  which  the  computational  model  has  not  been 
validated.”  They  are  the  primary  processes  for  quantifying  and  building  confidence  (or  credibility)  in 
numerical  models.  Verification  is  the  process  of  determining  that  a  model  implementation  accurately 
represents  the  developer’s  conceptual  description  of  the  model  and  the  solution  to  the  model.  It  is 
concerned  with  identifying  and  removing  errors  in  the  model  by  comparing  numerical  solutions  to 
analytical  or  highly  accurate  benchmark  solutions.  Validation,  on  the  other  hand,  is  concerned  with 
quantifying  the  accuracy  of  the  model  by  comparing  numerical  solutions  to  experimental  data.  It  is 
the  process  of  determining  the  degree  to  which  a  model  is  an  accurate  representation  of  the  real 
world  from  the  perspective  of  the  intended  uses  of  the  model.  In  short,  verification  deals  with  the 
mathematics  associated  with  the  model,  whereas  validation  deals  with  the  physics  associated  with  the 
model.  Verification  and  validation  are  processes  that  collect  evidence  of  a  model’s  correctness  or 
accuracy  for  a  specific  scenario;  thus,  V&V  cannot  prove  that  a  model  is  correct  and  accurate  for  all 
possible  conditions  and  applications,  but,  rather,  it  can  provide  evidence  that  a  model  is  sufficiently 
accurate.  Therefore,  the  V&V  process  is  completed  when  sufficiency  is  reached.  Defining  an 
acceptable  level  of  sufficiency  for  evaluation  of  the  accuracy  of  the  software  developer’s  responses 
will  be  decided  by  the  NATO  RTO  Task  Group  committee. 


12.8  SCOPE  OE  WORK  /  SCHEDULE  (DRAET) 

The  tentative  schedule  for  the  Phase  1,  Benchmarking  exercise  is  the  following: 

•  1  Jan  2016  -  Solicit  openly  for  developers  to  participate 

•  1  Apr  2016  -  Provide  vehicle(s)  data,  event  data,  and  validation  data  to  participants 

•  1  Aug  2016  -  Receive  participant  responses 

•  1  Sep  2016  -  Report  demonstration  results  to  NATO  RTO 

*Schedule  may  be  pulled  forward  to  meet  the  NATO  meeting  schedule  of  April  and  September  2016. 


12.9  CONCLUSIONS 

Since  Theme  7’s  responsibility  thus  far  has  been  to  come  up  with  a  path  forward  regarding 
evaluation  of  software  developer’s  responses,  no  real  conclusions  can  be  drawn  at  this  point.  Once 
the  Phase  I  “Tool  Benchmarking  V&V”  with  developers  has  been  completed,  the  larger  NATO  RTO 
Task  Group  committee  will  be  able  to  assess  the  state-of-the-art  and  determine  a  more  focused  path 


UNCLASSIFIED:  Distribution  Statement  A.  Approved  for  public  release;  distribution  is  unlimited 


148 


forward.  The  committee  will  then  continue  on  with  Phase  II  to  develop  NG-NRMM  standards 
version  1.0  and  associated  benchmarks  and  establish  the  basis  and  process  for  on-going  future 
development,  configuration  management,  and  tool  qualification.  Phase  II  will  be  much  larger  in 
scope  and  although  undefined  at  this  time,  will  most  likely  involve  a  full  scale  code  V&V.  Those 
efforts  will  determine  the  full  scope  of  the  NG-NRMM  standards  and  the  resulting  V&V  processes. 
They  will  most  likely  include  development  of  the  conceptual,  mathematical,  and  numerical  models; 
design  and  performance  of  V&V  experiments;  incorporation  of  independent  data  into  the  V&V 
process;  code  and  model  verification  efforts;  and  full  scale  code  evaluations.  The  expected  final 
deliverables  of  the  NATO  RTO  Task  Group  effort  will  be: 

•  A  set  of  standards  to  guide  the  implementation  of  the  NG-NRMM,  as  well  as  its  use  and 
management 

•  A  set  of  benchmarks  that  can  be  used  by  any  nation/developer  to  demonstrate  compliance 
with  NG-NRMM  standards 

•  An  identification  of  developer(s)  that  can  deliver  software  to  adequately  address  mobility 


UNCLASSIFIED:  Distribution  Statement  A.  Approved  for  public  release;  distribution  is  unlimited 


149 


Chapter  13  -  CONCLUSIONS  AND  RECOMMENDATIONS 

Jean  Dasch 


NATO  Exploratory  Team  148  (ET-148)  was  proposed  and  approved  in  the  spring  of  2014  with  the  goal  of 
evaluating  the  need  for  a  Next-Generation  NATO  Reference  Mobility  Model  (NG-NRMM).  The  current 
NRMM  is  a  simulation  tool  developed  in  the  1970s  hy  the  U.S.  Army  to  predict  the  capability  of  a  vehicle  to 
move  over  a  specified  terrain.  Due  to  improvements  in  simulation  capabilities  since  that  time,  the  ET’s  task  was 
to  evaluate  if  an  improved  model  could  be  developed.  To  enable  that  evaluation,  seven  theme  areas  were 
delineated  in  the  areas  of  Requirements;  Methodologies;  Stochastics;  Intelligent  Vehicles;  Tool  Choices;  Input 
Data  and  Output  Metrics;  and  Verification  and  Validation.  A  short  summary  of  the  results  from  each  theme  area 
are  covered  below. 

13.1  REQUIREMENTS 

The  goal  of  Theme  1  was  to  capture,  consolidate  and  summarize  the  mobility  modeling  capabilities  desirable  for 
the  NG-NRMM.  The  entire  membership  was  queried  as  to  the  pros,  cons,  and  missing  capabilities  of  the 
NRMM.  Erom  the  hundreds  of  items  submitted,  the  list  was  gradually  winnowed  down  to  requirements  for  a 
Near-Term  Solution  (Threshold)  and  for  a  Ear-Term  Solution  (Objective)  (Eigure  6-1).  The  Near-Term  Solution 
would  be  based  on  physics-based  models  such  as  Becker-Wong  rather  than  empirical  assessment.  The  Ear-Tern 
Solution  would  rely  on  more  advanced  Discrete  Element  Method  (DEM)  models  and  Einite  Element  Models 
(EEM)  requiring  high-performance  computers. 

The  NG-NRMM  would  include  larger  scale  terrains  with  variable  resolutions  dependent  on  the  area  covered. 
There  would  be  a  necessary  trade-off  between  computational  efficiency  and  model  fidelity.  Two  areas  that  were 
under  consideration  that  were  not  part  of  the  original  NRMM  were  Stochastics  or  Uncertainty  (Theme  3)  and 
Intelligent  Vehicles  (Theme  4). 


13.2  METHODOLOGIES 

The  NRMM  model  is  used  in  vehicle  design,  acquisition,  and  operational  planning.  The  vision  of  the 
Methodology  Theme  area  was  to  develop  an  Open-Architecture  model  with  a  Semi- Analytical  approach  most 
possible  in  the  short  time  frame  (Threshold)  with  a  long-term  goal  of  an  Analytical  Model  (Objective).  The 
Open  Architecture  would  provide  a  framework  for  modular,  interoperable  capabilities  with  the  simplest  form 
being  a  set  of  mobility  standards  or  specifications,  designated  as  NORMMS  for  NATO  Operational  Reference 
Mobility  Modeling  Standards.  The  NORMMS  framework  was  defined  as  a  modeling  and  simulation 
architectural  specification  that  promotes  standardization,  integration,  modular  interoperability,  portability, 
expansion,  verification  and  validation  of  vehicle-terrain  interaction  models. 

Other  recommendations  are  to  develop  a  requirements  dashboard,  Verification  and  Validation  benchmarks,  a 
software  assessment  matrix  and  to  follow  standards  similar  to  those  of  the  National  Agency  for  Einite  Element 
Methods  and  Standards  (NAEEMS). 
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13.3  STOCHASTICS 


This  theme  area  sought  to  describe  a  framework  for  a  stochastic  approach  for  mobility  predictions  over  large 
regions  that  could  be  integrated  into  NG-NRMM,  where  both  the  terrain  profile  and  vehicle-terrain  interaction 
play  a  key  role.  The  uncertainty  in  these  variables  leads  to  unreliable  model  results.  This  theme  area 
evaluated  the  stochastics  of  elevation  as  determined  by  remote  sensing,  and  the  physical  properties  of  the 
terrain  such  as  soil  cohesion  and  internal  friction  angle. 

A  framework  was  described  for  a  stochastic  approach  for  vehicle  mobility  prediction  over  large  regions  (>  5  x 
5  [Km^]).  In  this  framework,  a  model  of  the  terrain  is  created  using  geostatistical  methods.  The  performance 
of  a  vehicle  is  then  evaluated  while  considering  the  terrain  profile  and  the  vehicle -terrain  interaction.  In  order 
to  account  for  uncertainty,  Monte  Carlo  simulations  are  performed,  leading  to  a  statistical  analysis. 
Uncertainty  in  elevation  is  due  to  the  new  interpolated  terrain  model  to  a  higher  spatial  resolution  than  the 
original  DEM  (through  a  geostatistical  method  called  Ordinary  Kriging).  Uncertainty  in  soil  properties  is 
obtained  considering  the  variability  of  the  parameters  involved  in  the  well-known  Bekker-Wong  (BW)  model, 
rather  than  Cone  Index. 

The  algorithm  and  hardware  must  be  selected;  reduced  order  models  can  be  run  online  on  a  laptop,  whereas 
complex  models  could  require  offline  use  on  a  HPC.  Software  for  geostatistical  functions  would  be  required 
such  as  ArcGIS. 

13.4  INTELLIGENT  VEHICLES 

The  goal  of  this  theme  was  to  define  an  NG-NRMM  approach  and  requirements  to  assess  mobility  for 
intelligent  vehicles.  Intelligent  vehicle  technology  is  rapidly  evolving  and  NRMM  must  grow  and  adapt  with 
it.  Some  of  the  path-forward  questions  are  the  following: 

•  What  is  the  scope  of  intelligent  vehicles  to  consider? 

•  What  methods  to  address  and  priorities? 

•  What  tools  need  to  be  developed? 

•  What  benchmark  problems  should  we  pilot? 

During  the  next  phase  a  pilot  project  could  help  flesh  out  requirements,  challenges  and  gaps  for  intelligent 
vehicles.  This  pilot  would  show  sliding  levels  of  autonomy  under  multiple  scenarios  and  output  quantitative 
risk  and  performance,  leading  to  a  new  capability  development. 

13.5  TOOL  CHOICES 

The  goal  of  this  theme  was  to  identify  the  critical  elements  needed  in  an  NG-NRMM,  identify  potential  solutions 
throughout  the  technical  community  and  provide  a  robust  review  through  a  Request  for  Information.  Responses 
from  twelve  software  packages  were  evaluated  through  a  Combinatorial  Trade  Study  process.  This  effort 
demonstrated  that  tools  do  exist  from  commercial  and  academic  sources  that  meet  most  of  the  future  needs,  so  a 
major  development  effort  by  the  NATO  community  should  not  be  required. 

Accuracy  of  vehicle  system  performance  is  the  biggest  limitation  of  the  current  NRMM  which  is  empirically 
based.  Validated  physics-based  methods  will  potentially  be  an  improvement  over  NRMM.  The  strength  of  the 
physics-based  tools  varies.  Some  are  capable  and  have  been  thoroughly  validated  for  on-road  operation  and  yet 
only  limited  off-road  deformable  soils  work  has  been  accomplished.  Others  focused  primarily  on  off-road  soft 
soil  terrain  but  have  no  capability  for  determining  on-road  stability  and  associated  dynamic  control.  Furthermore, 
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many  of  the  potential  physics-based  tool  providers  are  not  familiar  with  the  existing  capabilities  of  NRMM, 
particularly  as  it  relates  to  developing  specific  terrains  appropriate  for  worldwide  deployment.  A  Verification 
and  Validation  exercise  is  required  to  evaluate  and  help  develop  the  existing  tools,  which  could  require 
substantial  funding. 

13.6  INPUT  DATA  AND  OUTPUT  METRICS 

The  goal  of  this  theme  was  to  define  the  inputs  and  output  requirements  that  will  inform  the  NG-NRMM  tool 
development/selection  process.  Seven  data  categories  of  inputs  were  designated:  vehicle;  terrain; 
environment/scenario;  operator;  human  factors;  intelligent  vehicle;  and  scale/resolution  modes.  Several  near- 
term,  stop-gap  solutions  were  described  that  were  developed  by  AMSAA  to  enhance  the  current  NRMM 
including  a  System  Level  Analysis  Mobility  Dashboard,  an  Urban  Maneuverability  Model  and  an  Optimal  Path 
Model. 

Future  challenges  will  include  the  following  areas:  develop  methodology  to  transform  high  resolution  satellite 
imagery,  remotely-sensed  GIS  data,  etc.  into  accurate  NG-NRMM  terrain  representations;  develop 
interoperability  standards  between  multi-body  vehicle  dynamic  simulations  and  commercial  GIS  software 
solutions;  and  pursue  multiple  levels  of  fidelity  solutions. 

13.7  VERIFICATION  AND  VALIDATION 

The  goal  of  Theme  7  was  to  provide  a  process  for  conducting  a  successful  tool  and  software  code 
Verification  and  Validation  (V&V)  program  on  NG-NRMM.  Plans  were  made  to  conduct  a  Phase  I 
Tool  Benchmarking  using  test  data  from  one  wheeled  and  one  tracked  vehicle  to  provide  a  common 
basis  for  evaluating  tool  capabilities.  Eight  software  developers  attended  the  NATO  meeting  in  Prague 
to  describe  their  capabilities  and  to  become  informed  of  the  future  V&V  plans. 

This  will  be  followed  by  a  Phase  II  to  develop  NG-NRMM  standards  version  1.0  and  associated 
benchmarks  and  to  establish  the  basis  and  process  for  on-going  future  development,  configuration 
management,  and  tool  qualification 
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Appendix  A  -  ET-148  TECHNICAL  ACTIVITY  PROPOSAL  (TAP) 


ACTIVITY 

REEERENCE 

NUMBER 

AVT-ET 

ACTIVITY  TITLE 

Next-Generation  NATO  Reference  Mobility  Model 
(NRMM)  Development 

APPROVAL 

TBA 

TYPE  AND  SERIAL 
NUMBER 

Exploratory 

Team 

START 

5/2014 

LOCATION(S)  AND  DATES 

In  conjunction  with  AVT  PBWs 

END 

4/2015 

COORDINATION  WITH  OTHER 
BODIES 

None 

NATO  CLASSIEICATION  OF 

ACTIVITY 

NU 

Non-NATO  Invited 
No 

PUBLICATION  DATA 

TM,  Misc 

NU 

KEYWORDS 

Mobility,  Ground  Vehicle,  NRMM 

A,1  BACKGROUND  AND  JUSTIFICATION  (RELEVANCE  TO  NATO): 

The  NATO  Reference  Mobility  Model  (NRMM)  is  a  simulation  tool  aimed  at  predicting  the  capability  of  a 
vehicle  to  move  over  specified  terrain  conditions.  NRMM  can  be  used  for  on-road  and  cross-country  scenarios, 
it  can  account  for  several  parameters  such  as  terrain  type,  moisture  content,  terrain  roughness,  vehicle  geometry, 
driver  capabilities,  etc. 

NRMM  was  developed  and  validated  by  the  U.S.  Army  Tank  Automotive  Research,  Development,  and 
Engineering  Center  (TARDEC)  and  Engineer  Research  and  Development  Center  (ERDC)  over  several  decades, 
and  has  been  revised  and  updated  throughout  the  years,  resulting  in  the  most  recent  version,  NRMM  II.  NRMM 
is  traditionally  used  to  facilitate  comparison  between  vehicle  design  candidates  and  to  assess  the  mobility  of 
existing  vehicles  under  specific  scenarios. 

Although  NRMM  has  proven  to  be  of  great  practical  utility  to  the  NATO  forces,  when  compared  to  modem 
modeling  tools  it  exhibits  several  inherent  limitations: 

•  It  is  based  on  empirical  observations,  and  therefore  extrapolation  outside  of  test  conditions  is  difficult  or 
impossible. 

•  It  is  heavily  dependent  on  in-situ  soil  measurements. 

•  Only  one -dimensional  analysis  is  possible;  lateral  vehicle  dynamics  are  not  considered. 

•  It  does  not  account  for  vehicle  dynamic  effects,  but  instead  only  considers  steady-state  condition. 

•  It  is  specific  to  wheeled/tracked  vehicles. 
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•  It  is  not  easily  implementable  within  modem  vehicle  dynamics  simulations. 

•  It  exhibits  poor  (or  poorly  understood)  inter-operability  and  inter-scalability  with  other  terramechanics  and 
soil  mechanics  models. 

•  It  is  only  suitable  for  mobility  analysis,  and  does  not  provide  auxiliary  outputs  (e.g.  power  efficiency 
analysis). 

The  proposed  exploration  is  vital  to  NATO’s  mission.  It  promises  to  enable  new  capabilities  in  the  design, 
modeling,  and  simulation  of  a  broad  class  of  vehicles.  These  modeling  capabilities  are  of  high  importance  to 
current  and  future  NATO  missions  because  they  have  the  potential  to  significantly  reduce  costs  and  improve 
performance.  The  new  tool  will  be  applicable  to  various  running  gear  morphologies,  including  conventional 
wheels  and  tracks,  and  more  novel  bio-inspired  limb  designs.  This  could  yield  a  new  paradigm  for  ground 
vehicle  mobility,  which  surpasses  traditional  analysis  based  on  NRMM’s  GO/NOGO  basis.  An  important  aspect 
of  modem  simulations  is  the  possibility  to  model  complex  vehicle  maneuvering  in  high  fidelity.  Relying  on 
High  Performance  Computing  (HPC),  it  will  be  possible  to  utilize  statistical  representations  of  terrain  profile  and 
properties  and  to  exploit  very  large-scale  Monte  Carlo  simulations  to  yield  rich  outputs  over  a  broad  parameter 
space. 


A.2  OBJECTIVE(S): 

This  scope  is  to  investigate  an  efficient  simulation-based  next-generation  NRMM.  Specifically,  the  proposed 

activity  will  focus  on  the  following  fundamental  scientific  objectives: 

•  Identify  scale -invariant  terrain  descriptions  for  representing  topographic  map  data  (obtained  at  various 
scales)  within  a  suitable  multi-body  dynamic  simulator.  This  will  enable  automated  analysis  of  regions  of 
interest,  given  heterogeneous  map  data  products  as  inputs. 

•  Develop  efficient,  automated,  parallelizable  experimental  design  methods  (i.e.  sampling  methods)  for 
extracting  metrics  of  interest  from  Monte  Carlo  simulations  of  the  multi-body  dynamic  simulator,  including 
mobility-related  metrics  and  auxiliary  metrics.  This  will  yield  rich  statistical  mobility-related  outputs  in  a 
computationally  efficient  manner,  which  will  allow  use  of  modem  HPC  resources. 

•  Explore  the  use  of  compact  representations  of  vehicle  dynamics  (i.e.  response  surface  methods  or  other 
approximation  methods)  within  the  multi-body  dynamic  simulator,  with  a  goal  of  further  reducing 
computational  cost. 

•  Establish  compact,  user-friendly  representations  of  output  metrics  that  capture  important  dependencies.  This 
will  yield  an  update  to  classical  “speed  made  good”  or  “go/no  go”  maps. 

A.3  TOPICS  TO  BE  COVERED: 

Modernizing  the  NRMM  involves  several  topics  of  effort: 

•  Identification  of  vehicle  -  terrain  interaction  models,  i.e.,  terramechanics  models,  that  balance  fidelity  with 
computational  efficiency. 

•  Development  of  in-si tu  and  online  measurement  tools  to  identify  required  terrain  parameters. 

•  Identification  of  the  type  and  form  of  desired  responses,  to  yield  rich  mobility  predictions  and  (ideally) 
useful  auxiliary  outputs. 
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•  Integration  of  terramechanics  models  into  modem  dynamic  simulation  software,  and  develop  efficient, 
automated  computation  tools,  which  will  ideally  enable  the  use  of  high  performance  computation 
techniques. 

•  Since  the  next-gen  NRMM  is  expected  to  he  extremely  computationally  intensive,  there  exists  a  need  to 
investigate  numerical  methods  to  improve  algorithmic  efficiency  and  automate  NRMM  output  generation, 
such  as  Monte  Carlo  sampling  techniques  and  stochastic  response  surfaces. 

A.4  DELIVERABLE  AND/OR  END  PRODUCT: 

The  Exploratory  Team  will  prepare  a  report  of  findings  and  recommendations  on  the  benefits  and  value  of  the 
Next-Generation  NATO  Reference  Mobility  Model  for  enhanced  vehicle  design  and  mobility  performance.  The 
report  will  also  detail  the  various  resources  required  and  committed  by  the  various  member  nations  to  develop 
this  model.  This  summary  report  will  detail  the  current  state-of-the-art  and  provide  recommendations  for  the 
next-gen  NRMM  that  will  be  more  predictive,  more  general,  and  more  scalable  than  the  current  NRMM. 

It  is  expected  that  the  findings  of  this  ET  will  lead  to  a  RTO  Task  Group  (RTG)  which  will  work  on  this 
cooperative  research  project  in  the  2015-2018  timeframe.  The  future  RTG  will  bring  together  experts  in  the  field 
from  all  NATO  and  supporting  nations  to  first  develop  the  technical  research  required  to  develop  the  next 
Generation  NRMM  model,  and  secondly  develop  computer  algorithms  to  rapidly  compute  and  automate  NRMM 
output  generation.  It  is  also  possible  that  one  or  more  RTO  Workshops  (RWS)  may  be  necessary  in  conjunction 
with  the  bi-annual  AVT  Meetings  to  focus  on  specific  aspects  of  the  challenges  facing  the  RTG.  A  Einal 
Technical  Report  is  expected  to  be  delivered  in  or  around  Oct  2018. 


A.5  TECHNICAL  TEAM  LEADER  AND  LEAD  NATION: 

Co-Chair:  Dr.  Paramsothy  Jayakumar  (U.S.  Army  TARDEC),  USA 
Co-Chair:  TBD 
Lead  nation:  USA 

AVT  Panel  Mentor:  Dr.  David  Gorsich  (U.S.  Army  TARDEC),  USA 


A.6  NATIONS  WILLING/INVITED  TO  PARTICIPATE: 

Canada,  Czech  Republic,  Estonia,  Erance,  Germany,  Italy,  Poland,  Romania,  Slovakia,  Turkey,  United 
Kingdom,  USA 


A.7  NATIONAL  AND/OR  NATO  RESOURCES  NEEDED: 

The  Exploratory  Team  will  need  meeting  space  during  AVT  Panel  Business  Weeks. 

Standard  support  for  a  Workshop  (RWS)  and/or  Specialists  (RSM)  meeting  and  Exploratory  Team.  This  will 
include: 
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•  National  support  for  the  Exploratory  Team  activity 

•  Technical  Evaluator  for  the  Workshop/Specialists  meeting 

•  Distrihution  of  Workshop/Specialists  announcements 

•  Publication  of  the  Proceedings  of  the  Workshop/Specialists  meeting  on  the  RTO  Wehsite 

•  Publication  of  the  Exploratory  Team  Report 


A.8  RTA  RESOURCES  NEEDED: 

Standard  support  for  a  Workshop  (RWS)  and/or  Specialists  (RSM)  meeting  and  Exploratory  Team. 
This  will  include: 

•  Technical  Evaluator  for  the  Workshop/Specialists  meeting 

•  Distribution  of  Workshop/Specialists  announcements 

•  Publication  of  the  Proceedings  of  the  Workshop/Specialists  meeting  on  the  RTO  Website 

•  Publication  of  the  Exploratory  Team  Report 
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Appendix  B  -  FINAL  REPORT  FOLLOWING  ET-I48  MEETING  IN 

BELGIUM 


Some  Thoughts  on  the  Development  of  the 
Next-Generation  NRMM 

J.Y.  Wong 

Vehicle  Systems  Development  Corporation 
Ottawa,  Ontario,  Canada 


1.  Introduction 

The  NATO  AVT-ET-148  meetings  were  held  in  Brussels,  October  13-17,  2014,  to  discuss  the 
framework  within  which  the  next-generation  NRMM  may  he  developed.  The  discussions  focused  on  its 
goals,  requirements,  methodology,  input  and  output  metrics,  and  related  topics.  To  provide  the  necessary 
background  information  for  discussions,  the  following  presentations  on  various  themes  were  made  at  the 
meetings: 

(A) .  Next-Gen  NRMM  Goals  and  Themes,  by  Dr.  P.  Jayakumar,  TARDEC 

(B) .  Theme  1:  Requirements,  by  Jody  Priddy  and  Wendell  Gray,  ERDC 

(C) .  Theme  2  a:  Methodology,  by  Dr.  Mike  McCullough,  BAE 

(D) .  Theme  2  b:  Methodology-Stochastics,  by  Dr.  Karl  lagnemma,  MIT 

(E) .  Theme  2  c:  Tool  Choices,  by  Henry  Hodges,  NATC 

Mobility  Tool  Choices  of  Germany  and  Erance,  by  Dr.  Michael  Hoenlinger,  Germany 
(E).  Theme  2  d:  Methodology  -  Intelligent  Vehicles,  by  Dr.  Karl  lagnemma,  MIT 

(G) .  Interim  Report  of  the  Project  “Evaluation  of  NTVPM  for  Assessing  Tracked  Vehicle  Cross- 
Country  Performance”,  by  Dr.  J.Y.  Wong,  VSDC.  The  project  is  sponsored  by  TARDEC 

(H) .  Theme  3:  Input  Data  and  Output  Metrics,  by  James  Ngan  and  Brian  Wojtysiak,  AMSAA 

(I) .  Theme  4:  All  Other  Items,  by  Dr.  P.  Jayakumar,  TARDEC 

Inspired  by  these  presentations  and  on  reflection  of  the  ensuing  discussions,  some  of  the  thoughts  on  the 
development  of  the  next-generation  NRMM  were  offered  in  this  brief  report  by  the  author,  as  consultant  to 
the  NATO  Science  and  Technology  Organization  (STO),  Collaboration  Support  Office  (CSO).  This  brief 
report  is  intended  to  summarize  the  issues  that  should  be  addressed  in  the  development  of  the  next- 
generation  of  NRMM.  It  is  not,  however,  intended  to  provide  any  recommendation  for  its  execution.  This 
can  only  be  made  after  an  in-depth  analysis  and  evaluation  of  all  the  issues  involved,  which  is  beyond  the 
scope  of  the  tasks  stipulated  in  the  Consultancy  Contract  with  NATO  STO  CSO  (CP-AVT-ET- 148- 14-807). 

Goals 

It  is  suggested  that  the  primary  goals  of  the  next-generation  NRMM  be: 

Providing  military  agencies  of  NATO  countries  with  advanced  tools 

(a) ,  to  evaluate  ground  vehicle  candidates  in  sufficient  detail  in  the  procurement  process; 

(b) .  to  perform  operational  planning  for  the  deployment  of  military  ground  vehicles  in  the  field; 

(B).  providing  industry  with  a  reference  in  the  development  of  ground  vehicles  to  meet  military 
requirements. 

The  next-generation  NRMM  should  incorporate  the  latest  advancements  in  modeling  and  simulations  of 
ground  vehicles,  which  include  but  are  not  limited  to  advances  in  the  analysis  of  the  mechanics  of  vehicle- 
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terrain  interaction,  terrain  characterization,  simulation  techniques,  and  military  ground  vehicle  technologies. 


Requirements 

The  requirements  for  the  next-generation  NRMM  include  hut  are  not  limited  to  the  following: 

(A) ,  physics-hased,  that  is,  based  on  the  understanding  of  the  physical  nature  of  vehicle -terrain 
interaction  and  on  the  detailed  analysis  of  its  mechanics; 

(B) .  capability  in  evaluating  military  ground  vehicle  performance  and  behavior  in  three  dimensions; 

(C) .  capability  in  modeling  military  ground  vehicle  performance  and  behavior  on  both  rigid  surfaces  and 
deformable  terrains;  measurement  and  characterization  of  deformable  terrain  behavior  be  consistent  with 
Requirement  3  (A); 

(D) .  capability  of  simulating  legged  vehicles,  robotic  vehicles,  and  intelligent/autonomous  vehicles,  in  addition 
to  conventional  wheeled  and  tracked  vehicles; 

(E) .  capability  in  modeling  ground  vehicle  performance  and  behavior  equipped  with  various  sub-systems, 
including  but  not  limited  to  antilock  braking  systems,  traction  control  systems,  dynamic  stability  control 
systems,  active/semi-active  suspensions,  and  powertrain  systems,  as  well  as  vehicle  fuel  economy; 

(F) .  capability  in  integrating  driver  models  in  simulations  of  ground  vehicle  performance  and  behavior; 

(G) .  sufficient  accuracy  (fidelity)  to  enable  meaningful  differentiation  of  the  performance  and  behavior  of 
military  ground  vehicles  of  various  configurations  and  designs,  in  accordance  with  the  Goal  noted  in  2  (A) 
(a); 

(H) .  modular  structure  to  enable  the  expansion  of  its  capabilities  to  meet  new  challenges  in  the  future; 

(I) ,  user-friendly  in  input  and  output  and  ease  of  its  operations; 

(J) .  verification  and  experimental  validation  of  its  predictive  capabilities  on  rigid  surfaces  and  on 
representative  deformable  terrains  (such  as,  fine-  and  coarse-  grained  soil,  muskeg  (organic  terrain),  and  snow- 
covered  terrain). 

4.  Implementation 

In  the  development  of  the  next-generation  NRMM,  the  implementation  issues  to  be  considered  include 
but  are  not  limited  to  the  following: 

(A) ,  investigating  the  feasibility  of  establishing  a  framework  (or  “backbone”)  for  the  next-generation 
NRMM,  with  which  various  modules  may  be  connected  with  standardized  input  formats  and  from  which 
specific  output  with  standardized  formats  may  be  obtained.  The  framework  is  a  computer  simulation 
architectural  specification  applicable  to  the  full  range  of  ground  vehicle  geometric  scales  that  promotes 
standardization,  integration,  interoperability,  expansion,  verification,  and  validation  of  vehicle -terrain 
interaction  models  at  multiple  levels  of  analytical  and  numerical  resolution  [1]; 

(B) .  examining  whether  the  framework  be  based  on  commercial  software  or  be  established  with  specially 
developed  codes,  taking  into  account  the  costs,  security,  legal  implications,  sustainability,  and  training; 

(C) .  various  aspects  of  ground  vehicle  performance  and  behavior  to  be  predicted  using  separate 
modules,  including  but  not  limited  to: 

(a) ,  on-road  performance  prediction  module; 

(b) .  cross-country  performance  prediction  module  (since  the  widely  accepted  practice  is  to  evaluate 
cross-country  performance  under  steady-state  operating  conditions,  it  is  much  more  efficient  to  obtain 
output  metrics  through  solving  a  set  of  vehicle  and  sub-system  dynamic  equilibrium  equations  than  by  time 
integration  of  a  set  of  equations  of  motion); 

(c) .  ride  quality  prediction  module  for  rigid  surfaces  and  deformable  terrains; 

(d) .  handling  characteristics  prediction  module  for  rigid  surfaces  and  deformable  terrains  (including  urban 
maneuverability) ; 

(e) .  obstacle  crossing  performance  prediction  module;  (f).  amphibious  capability  prediction  module. 
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(D) .  evaluating  the  methodologies  for  measuring  and  characterizing  deformable  terrain  behavior  in 
accordance  with  Requirement  3  (A)  (including  methodologies  based  on  the  cone  penetrometer,  bevameter, 
and  traditional  devices  utilized  in  civil  engineering  soil  mechanics); 

(E) .  incorporating  uncertainties,  stochastic  and  sampling  methods  into  terrain  data  acquisition  and 
characterization,  as  well  as  the  propagation  of  uncertainty  of  terrain  input  to  output  metrics; 

(F) .  template -based  input  for  vehicle  sub-systems; 

(G) .  establishing  output  metrics,  such  as,  mobility  map,  mobility  profile,  gradeability,  tractive  performance, 
and  fuel  economy  for  cross-country  operations;  acceleration  time  and  distance,  braking  distance, 
gradeability,  and  fuel  consumption  for  on-road  operations;  weighted  root  mean  squared  acceleration, 
absorbed  power,  instantaneous  peak  acceleration,  and  frequency  response  for  ride  quality;  minimum 
turning  radius,  yaw  velocity  response,  lateral  acceleration  response,  curvature  response  for  handling 
characteristics,  etc.  [2]; 

(H) .  utilizing  high-power  computing  (HPC)  resources,  if  necessary. 

Research  in  Support  of  Implementation 

Research  in  support  of  the  initial  phase  of  development  of  the  next-generation  NRMM  includes  but  is  not 
limited  to  the  following: 

(A) ,  in  simulating  ride  quality  of  vehicles  over  deformable  terrains,  the  usual  practice  is  to  use  springs  and 
dampers  to  model  the  terrain.  In  essence,  the  terrain  is  assumed  to  be  a  visco-elastic  medium.  In  accordance 
with  Requirement  3  (A),  the  plastic  deformation  of  deformable  terrains  should  be  considered,  so  that  the 
modification  to  the  terrain  profile  due  to  vehicle-terrain  interaction  is  properly  taken  into  account; 

(B) .  models  for  simulating  maneuverability  of  ground  vehicles,  including  tracked  vehicles,  on  non- 
deformable  surfaces  have  been  established  [2,  3] .  In  accordance  with  Requirement  3  (A),  the  mechanics 
of  vehicle -terrain  interaction  during  maneuvers  on  deformable  terrains  should  be  examined. 

Priority 

In  the  development  of  the  next-generation  NRMM,  priority  should  be  established  in  accordance  with  the 
urgency  of  needs  and  their  potential  impacts.  In  the  initial  phase  of  its  development,  the  following  should  be 
considered: 

(A) ,  the  establishment  of  the  framework,  as  noted  in  4  (A),  is  key  to  the  development  of  the  next-generation 
NRMM  and  should  be  given  top  priority; 

(B) .  cross-country  performance  is  one  of  the  focuses  in  the  evaluation  of  military  ground  vehicle 
mobility.  In  the  current  NRMM,  the  cross-country  performance  prediction  sub-module  is  entirely  based  on 
empirical  relations.  This  indicates  that  the  development  of  physics-based,  cross-country  performance 
prediction  methodology  should  be  given  priority; 

(C) .  in  the  current  NRMM,  there  is  no  provision  for  evaluating  the  handling  characteristics  of  ground 
vehicles.  This  suggests  that  the  development  of  maneuverability  prediction  methodology  also  be  given 
priority. 

7.  Collaboration 

Collaboration  with  professional  organizations  in  the  field  of  vehicle  mobility,  such  as  the  international 
Society  for  Terrain- Vehicle  Systems  (ISTVS),  would  be  useful.  The  collaboration  may  be  in  the  form  of 
organizing  special  workshops  and/or  forums  at  ISTVS  international  or  regional  conferences,  at  which 
advice  of  experts  may  be  sought  or  topics  of  interest  may  be  discussed. 
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Appendix  C  -  INITIAL  TEAM  SURVEY 


C.l  WHAT  ARE  THE  THINGS  THAT  YOU  LIKE  ABOUT  NRMM? 

Canada,  William  Mayda 

•  not  just  a  pure  "mobility"  tool.  Includes  other  human  factors 

•  relatively  good  correlation  when  soil  conditions  known 
Romania,  Ticusor  Ciobotam 

•  Exhaustive  covering  wheeled  and  tracked  vehicles  from  small  robot  to  tanks  sizes 

•  Fast  and  facile  method  for  soil  characteristics 

•  Impressive  experimental  sustentation 

•  Allows  simulations/predictions  for  expensive  test  (mobility  on  soft  soils,  suspensions  characteristics) 

•  Used  by  several  NATO  countries 
USA,  Dave  Gunter 

•  Provides  measure  of  mobility  performance  in  "Operational"  terms 

•  Portability  (desktop  capable) 

•  Runs  quickly 

•  Easy  to  develop  models 
USA,  Karl  lagnemma 

•  The  ambition  to  model  multiple  effects  related  to  terrain,  environment,  vehicle,  and  operator 

•  A  clear,  unambiguous  output  metric 
USA  Mike  McCullough 

•  Stable  mobility  metrics  and  criteria  creates  a  level  playing  field  for  use  in  trade  studies 

•  Most  metrics  have  a  trace-able  theory  that  enables  linkage  from  performance  results  to  design  attributes 

•  Available,  open  source  and  supported  for  use  by  industry 
USA,  Jody  Priddy 

•  NRMM  is  currently  the  only  available  modeling  and  simulation  (M&S)  product  that  can  realistically 
quantify  ground  vehicle  mobility  based  on  terrain  accessibility  and  maximum  attainable  speeds  for 
comparative  force  projection  assessments  of  military  vehicles  via  rational  consideration  of  the  vehicle's 
mission,  design  characteristics,  and  actual  terrain  characteristics  around  the  globe. 

•  One  of  its  key  strengths  originates  from  the  methods  used  to  compute  force  projection  metrics  by  integrating 
engineering-level  (i.e.,  proving  ground  type)  performance  capabilities  on  different  terrain  features  with 
geo-specific  quantifications  of  the  types  of  terrain  feature  interactions  that  will  occur  in  different  theaters  of 
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operation  around  the  world.  Metrics  associated  with  fundamental  engineering  level  performance  tests  are 
very  important  for  sound  decision  making  in  ground  vehicle  design,  hut  there  is  also  a  critical  need,  which 
NRMM  fulfills,  to  extend  engineering-level  performance  metrics  heyond  controlled  proving  grounds  and 
into  force  projection  metrics  that  quantify  real-world,  mission  based,  operational  capabilities. 

•  Another  key  strength  of  NRMM  is  the  viable  nature  of  the  underlying  models  and  relationships  for 
achieving  usable  force  projection  capability  assessments  in  a  reasonable  amount  of  time  without  a 
requirement  for  excessive  information  on  vehicle  and  terrain  characteristics  that  can  be  highly  restricted  or 
not  realistically  attainable.  Both  the  vehicle  and  terrain  characteristics  required  for  NRMM  are  robust  in 
scope,  yet  very  attainable. 

•  An  additional  key  strength  is  the  comprehensive  nature  of  NRMM  from  a  terrain  perspective,  especially  for 
mobility  performance  in  non-urban  and  off-road  environments.  It  can  currently  account  for  the  influence  of 
most  major  soil,  snow,  and  ice  ground  surface  conditions  (to  include  rainfall  induced  slipperiness  effects  on 
soils),  varying  slope  grades,  rough  undulating  terrain  surfaces,  discrete  shock  inducing  ground  obstacles,  dry 
and  water-filled  linear-feature  gaps,  vegetation  and  other  override  resisting  obstacles,  visibility  restricting 
terrain  features,  and  general  speed-limiting  features  of  road  networks. 

•  Finally,  NRMM  is  free  software  for  all  NATO  end-users  who  have  access.  End-users  incur  no  hefty  upfront 
purchasing  costs  or  recurring  maintenance  costs,  both  of  which  are  typical  for  most  commercial  engineering 
software  products.  In  the  case  of  NRMM,  development  and  maintenance  costs  of  the  software  products  and 
the  unique  embedded  M&S  knowledge  are  funded  through  government  research  and  development 
investments,  and  the  software  is  freely  distributed  for  use  in  government  purposes  only. 

USA,  Brian  Wojtysiak 

•  Quick  run  times 

•  Allows  us  to  support  Army  studies  involving  multiple  vehicles  under  relatively  short  deadlines  with 
an  appropriate  level  of  fidelity 

•  Assesses  the  combined  effects  of  a  variety  of  off-road  challenges  (soil  strength,  grades,  obstacles, 
vegetation,  ride  &  shock  tolerances,  weather  conditions,  human  factors,  etc) 

•  Provides  diagnostic  reason  codes  to  help  understand  results 

•  Empirical  relations  (i.e.  VCI  vs.  drawbar/resistance)  that  provide  a  level  of  self-validation 

•  Excellent  item/system-level  performance  estimation  tool.  One  of  the  only  tools  that  can  be  used  to 
conduct  wheeled  and  tracked  vehicle  off-road  mobility  analysis. 

•  The  effects  of  sub-system  design  changes  can  be  rapidly  assessed. 

•  Provides  strong  capability  to  execute  comparative  mobility  analysis  (including  backwards  comparability). 

•  NRMM  outputs  can  be  represented  with  maps  (speed  maps,  speed  comparison  maps.)  for  better  visualization 
/  comparison  (if  digital  terrain  file  available).  Although  this  process  can  be  time  consuming  and 
cumbersome. 

C.2  WHAT  ARE  THE  THINGS  THAT  YOU  DISLIKE  ABOUT  NRMM? 

Canada,  William  Mayda 

•  lack  of  friendly  user  interface 
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•  inability  to  extrapolate  beyond  existing  vehicle  types/weights 

•  inability  to  accommodate  new  and  novel  drivetrains 
Romania,  Ticusor  Ciobotam 

•  Data  input/output,  software  running 

•  Lack  of  friendly  GUI  for  input/output  data 

•  Lack  of  modules  covering  the  steering  of  the  tracked  vehicles 
USA,  Dave  Gunter 

•  No  error  handling  (crashes  when  data  entered  incorrectly  with  no  message  indicating  where  the  error  came 
from) 

•  Impossible  to  verify  many  of  the  predictions  through  test  (Mission  Rating  Speeds,  %NOGO,  etc.) 

•  2D-dynamics 

•  Not  possible  to  evaluate  modem  technologies  (active/semi  active  suspensions,  esc,  abs,  etc.) 

•  Simple  tire  model 

•  Small  portion  of  globe  incorporated  (areal  terrain  maps  need  to  be  expanded) 

•  Split  Mu 

•  No  Braking 

•  No  rocky  evaluation  capabilities 

•  Urban  maneuverability 
USA,  Karl  lagnemma 

•  Its  reliance  on  ad  hoc  correction  factors  to  model  the  effects  of  many  distinct  effects,  which  likely  leads  to 
substantial  uncertainty  in  the  resulting  output 

•  Its  lack  of  representation  of  output  uncertainty  levels,  making  it  difficult  to  assign  confidence  to  the  output 
USA  Mike  McCullough 

•  Ride  Quality  metric  needs  significant  updates 

•  3D  vehicle  multibody  dynamics  models  that  are  more  precisely  representative  of  vehicle  designs 
(must  include  flexible/deformable  bodies  to  be  general) 

•  3D  Deformable  terrain  in  the  simulations 

•  Terrain  specification  in  mission  profiles 

•  Spectral  content 

•  material  response,  i.e.,  soil  type  and  moisture  content 

•  Ergodic  and  stationary  sample  lengths  w.r.t.  ride  quality  response  parameters  (accounting  for 
skid  plate  and/or  spider  contact  events) 
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•  Driver  feedback  loop  model  for  speed  and  direction  control  of  3D  vehicle  dynamic  model 

•  Automated  iterative  loops  for  6watt  and  2.5G  speed  limits 

•  access  to  intermediate  and  lower  level  results  plots  such  as  speed  vs  power,  acceleration 

•  Obstacle  crossing  metrics  need  significant  updates 

•  3D  vehicle  multibody  dynamics  models  that  are  more  precisely  representative  of  vehicle  designs 

(must  include  flexible/deformable  bodies  to  be  general) 

•  3D  Deformable  terrain  with  embedded  hard  obstacles  in  the  simulations 

•  Rubble  pile  definition  and  standardization 

•  could  include  dynamic  rubble 

•  library  of  obstacles  that  are  selectable  and  tailorable  to  vehicle  and  mission  requirements 

•  amphibious  operations  obstacles 

•  stream/lake  fording 

•  surf  zones  including  rocky  shores 

•  ship  launch 

•  Needs  powertrain  performance  on  slopes  and  fuel  economy/range 

USA,  Jody  Priddy 

•  The  biggest  weakness  of  the  current  version  of  NRMM  is  the  dated  nature  of  the  software  code,  which  leads 
to  nonuser  friendliness  and  a  lack  of  modularity  for  ease  of  upgrades  and  variations.  The  development  and 
maintenance  investments  over  time  for  NRMM  have  largely  been  piecewise  and  project  focused,  with  no 
formalized  funding  process  identified  within  NATO  or  the  contributing  nations  specifically  for  software 
maintenance  and  updates.  There  have  been  research  and  development  investments  in  unique  embedded 
knowledge  and  capabilities  for  NRMM  by  contributing  nations,  but  a  lack  of  funding  directed  solely  at 
software  maintenance  purposes  has  resulted  in  the  current  outdated  state  of  the  software.  It  is  important  that 
formalized  software  maintenance  strategies  be  pursued  to  ensure  that  future  versions  of  the  NRMM  software 
can  be  kept  up-to-date  in  terms  of  computing  standards  and  capabilities. 

•  NRMM  does  not  currently  model  the  influence  of  active  traction  control  systems  such  as  anti-lock  braking 
(ABS),  automatic  brake  modulation  (ABM),  or  electronic  stability  control  (ESC).  Active  controller-based 
systems  for  traction  can  provide  significant  benefits  for  on-road  stability  and  performance,  but  their  effects 
on  off-road  performance  can  actually  be  detrimental  and  must  be  quantified  for  a  complete  assessment  of  a 
vehicle’s  performance  capabilities.  NRMM  currently  assumes  that  each  traction  element  (e.g.,  wheel,  track, 
etc.)  is  either  fully  unpowered  or  powered  (i.e.,  towed  or  driven  mode),  where  it  is  assumed  that  there  is 
ample  torque  to  fully  mobilize  all  of  the  traction  available  from  the  terrain  for  the  powered  case.  The 
influence  of  active  traction  control  systems  on  performance  could  be  modeled  in  NRMM  with  appropriate 
upgrades  to  eliminate  this  binary  assumption. 

•  NRMM  does  not  currently  model  the  influence  of  active  suspension  systems.  Active  suspension  systems  are 
a  future  technology  with  great  potential  to  produce  improvements  in  off-road  performance.  More  robust 
vehicle  dynamics  software  products  are  needed  for  modeling  active  suspension  systems  prior  to  the 
development  of  physical  prototypes.  Incorporation  of  controller  logic  algorithms  in  the  current  vehicle 
dynamics  preprocessor  VEHDYN  (a  relatively  “light  weight”  2-D  simulation  tool)  could  largely  overcome 
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the  associated  limitations,  but  integration  of  controller  M&S  and  full-featured  3-D  vehicle  dynamics 
simulation  tools  is  achievable  and  would  likely  provide  the  best  overall  capability  improvements  for 
NRMM. 

•  The  empirical  nature  of  the  current  vehicle-terrain  interaction  relationships  results  in  one  of  the  key  strengths 
of  NRMM  since  the  correlation  relationships  have  been  robustly  founded  on  large  quantities  of  physical 
measurements  with  vehicles  and  single  traction  elements  that  ensure  realistic  predictions  of  force  projection 
capabilities,  but  the  resulting  total  dependence  on  physical  test  data  to  derive  these  terramechanics 
relationships  also  results  in  a  key  weakness  for  NRMM  due  to  a  continuing  requirement  for  complex  and 
expensive  physical  testing.  The  empirical  relationships  provide  good  prediction  confidence  for  typical 
ground  vehicles,  not  only  because  of  the  robust  underpinning  data,  but  also  because  of  their  underlying 
physics  basis,  which  derives  from  consideration  of  the  controlling  physical  interactions  at  the  ground 
interface  involved  in  traction,  motion  resistance,  sinkage,  etc.  However,  correlation  relationships  will 
always  be  limited  in  applicability  to  the  empirical  range  of  the  underlying  data  and  the  bounding 
assumptions  behind  the  relationships,  which  demands  continuous  consideration  of  new  performance  data  to 
ensure  or  expand  the  applicability  of  the  terramechanics  relationships  to  evolving  and  atypical  vehicle 
designs.  The  terramechanics  relationships  in  NRMM  essentially  predict  the  response  characteristics  of 
terrain  to  loadings  imposed  by  ground  vehicles,  where  the  terrain  response  characteristics  typically  limit  the 
mobility  performance  of  military  vehicles.  Modeling  terrain  response  characteristics  through  numerical 
simulations  that  quantify  the  physics  of  stress  and  deformation  propagation  within  terrain  media  (e.g.,  soil, 
snow,  ice,  vegetation  obstacles,  etc.)  has  historically  presented  overly  formidable  challenges  that  have 
precluded  their  use  over  empirical  correlation  approaches,  but  recent  advancements  in  numerical  methods 
and  high  performance  computing  capabilities  are  now  beginning  to  offer  real  promise  for  enhancing, 
expanding,  or  replacing  physical  testing  with  virtual  performance-knowledge  generators. 

USA  Brian  Wojtysiak 

•  The  user  interface  (text  files  and  command  line)  is  not  user-friendly.  (AMSAA  is  currently  developing  a 
user  interface  “wrapper”  to  address  this  issue). 

•  Terrain  data  is  old,  not  up-to-date  and  new  terrains  cannot  be  easily  built  from  geospatial  data. 

•  It  would  be  nice  to  be  able  to  execute  with  less  data  fidelity  (especially  with  “red”  systems  where  there  is 
often  little  to  no  data  availability). 

•  Empirical  relations  limit  extrapolation  and  validity  of  assessing  future  technologies  making  it  difficult  to 
incorporate  new  vehicle  technologies  unless  the  analyst  can  identify  the  impacts  on  certain  vehicle  sub¬ 
systems. 

•  Statistical  outputs  and  speed  profiles  -  do  not  inform  mission  operations  -  (e.g.  the  mission  /  route  planning 
context). 

•  Statistical  output  does  not  consider  accessibility  -  e.g.  a  NTU  may  be  represented  as  “Go”;  however  the 
entire  NTU  is  surrounded  by  “NoGo”  terrain  and  therefore  is  inaccessible.  To  correct  this  issue,  additional 
spatial  analysis  post-processing  is  needed. 

•  The  vehicle  configuration  used  in  Obsmod  submodule  does  not  represent  the  actual  vehicle  configuration. 

•  Mobility  for  on  and  off  road  are  traditionally  evaluated  separately. 

•  Outdated  interface  for  input  and  output  files  (the  VEHDYN  preprocessor  can  be  particularly  problematic). 

•  Outputs  (i.e.  VXX  speeds)  can  be  difficult  to  understand  for  non-technical  personnel. 
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•  Lack  of  validation  with  NRMM  updates. 

•  Requirement  for  some  input  curves  (i.e.  ride  &  shock)  to  he  continually  decreasing  -  this  is  not  always  the 
case  in  real  world  due  to  resonances,  suspension  characteristics,  etc.  (e.g.  in  reality  they  are  not  “smooth” 
curves  -  “real-world”  data  may  have  “spikes”  to  account  for  this  type  of  behavior). 

•  Current  method  for  determining  No-Go  reason  codes  could  he  improved  -  for  example,  there  could  he 
multiple  reasons  for  No-Go,  hut  currently  only  one  reason  is  revealed  with  the  current  algorithms. 

•  Obstacle  No-Go  restricted  by  the  slightest  of  clearance  interference  -  doesn’t  represent  the  ability  of  the 
vehicle  to  override  the  obstacle  with  vehicle  horsepower. 

C.3  WHAT  ARE  YOUR  REQUIREMENTS  FOR  THE  NEXT-GENERATION 
NRMM? 

Canada,  William  Mayda 

•  Enhanced  user  interface 

•  Enhanced  graphical  output  (graphs,  charts,  visuals  etc.) 

•  Add  on  modules  for  unique  soil  conditions  (soft  soil,  snow  etc.)  with  physics  base 

•  "Lite"  version  that  would  allow  non-trained  users  to  vary  selected  parameters  easily  (perhaps  power,  weight 
etc.)  without  requiring  in-depth  knowledge.... quick  "what  if  scenarios 

Canada,  J.Y.  Wong 

•  In  the  development  of  the  Next-Gen  NRMM,  the  methodology  for  assessing  the  cross-country  performance 
should  be  given  priority.  The  reasons  for  this  are  well  articulated  in  Dr.  Jayakumar's  presentation  on  the 
inherent  limitations  of  the  current  version  of  NRMM. 

•  In  the  discussions  of  the  objectives  of  the  Next-Gen  NRMM,  perhaps  the  following  issues  should  be  given 
sufficient  attention: 


•  The  evaluation  of  vehicle  candidates,  from  the  cross-country  performance  perspective,  using  the 
current  version  of  NRMM  is  based  on  a  limited  number  of  criteria,  such  as  "go/no  go,”  "maximum 
possible  speed  (speed-made-good),”  etc.  Should  the  number  of  criteria  be  expanded  to  include  other 
factors,  such  as  efficiency? 

•  The  level  of  fidelity  at  which  the  Next-Gen  NRMM  is  aiming  should  be  carefully  considered,  in 
relation  to  the  proposed  time  frame  and  the  resources  available.  Eor  instance,  should  it  be  aiming  at 
replicating  vehicle  performance/behavior  in  the  field  in  detail  or  providing  a  simulation  tool  for 
evaluating/comparing  vehicle  candidates  on  a  relative,  yet  well-founded,  basis? 

Czech  Republic,  Neumann  Vlastimil 

•  Improvements  (definition)  of  terrain 

•  Utilization  of  simulating  technologies  in  process  of  vehicle  mobility  evaluation  (obstacles  negotiation) 

Estonia,  Kersti  Vennik 
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Prioritized  requirements  (objectives)  list  for  next-gen  NRMM: 

•  first  of  all  I  think  the  new  NRMM  should  be  easy  to  use  and  to  install; 

•  it  should  work  in  non-soil  scientist  mode,  i.e.  with  easy  No-Go  and  Slow-Go  estimation  option,  as  well  in 
terramechanic  specialist  mode,  where  more  detail  and  parameters  about  soil  as  well  vehicle  can  be  inserted 
and  modeled  with  different  soil-vehicle  interaction  models  (models  based  on  RCI  values,  models  based  on 
soil  strength  (internal  friction,  cohesion)  values,  etc.); 

•  the  modeling  output  should  be  in  digital  map  form  and  Open  Geospatial  Consortium  standards  for  the  digital 
maps  should  be  used,  so  that  final  results  could  be  loaded  to  different  CIS  and  C2  systems; 

•  possible  modeling  outputs  should  be: 

•  off-road  speed  estimation  for  particular  vehicle, 

•  rut  depth  estimation  for  first  and  for  0th  pass  for  particular  vehicle, 

•  soil  susceptibility  to  increase  of  moisture 

•  moving  possibilities  in  thawing  soil  situation  as  well  as  for  different  depth  of  snow  situation. 

Germany,  Michael  Hoenlinger 

From  development  perspective  I  would  prioritize  the  (TAP)  objectives  as  follows: 

•  Identify  scale -invariant  terrain  descriptions  for  representing  topographic  map  data  (obtained  at  various 
scales)  within  a  suitable  multi-body  dynamic  simulator.  This  will  enable  automated  analysis  of  regions  of 
interest,  given  heterogeneous  map  data  products  as  inputs. 

•  Develop  efficient,  automated,  parallelizable  experimental  design  methods  (i.e.  sampling  methods)  for 
extracting  metrics  of  interest  from  Monte  Carlo  simulations  of  the  multi-body  dynamic  simulator,  including 
mobility-related  metrics  and  auxiliary  metrics.  This  will  yield  rich  statistical  mobility-related  outputs  in  a 
computationally  efficient  manner,  which  will  allow  use  of  modem  HPC  resources. 

•  Explore  the  use  of  compact  representations  of  vehicle  dynamics  (i.e.  response  surface  methods  or  other 
approximation  methods)  within  the  multi-body  dynamic  simulator,  with  a  goal  of  further  reducing 
computational  cost. 

•  Establish  compact,  user-friendly  representations  of  output  metrics  that  capture  important  dependencies.  This 
will  yield  an  update  to  classical  “speed  made  good”  or  “go/no  go”  maps. 

Another  approach  could  be  to  establish  an  interface  between  NRMM  and  MBS  software.  The  advantage  is  that 

both  software  systems  could  be  updated  and  optimized  independent  and  only  the  interface  has  to  be  adapted. 

Romania,  Ticusor  Ciobotam 

Objectives: 

•  Requirements  for  a  friendly  GUI 

•  Conceptual  framework  for  dealing  with  steering 

•  Evaluation  of  the  impact  of  new  technologies  on  the  NRMM  modules  (hybrid,  or  electric  traction,  skid 
steering  for  wheeled  vehicles) 
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USA,  Dave  Gunter 


•  Need  to  research  testable  mobility  metrics. 

•  Need  to  research  rationale  for  asymmetric  terrains  (how  to  quantify  asymmetry,  and  why  it's  needed). 

•  Need  to  research  terrain  roughness  index  (for  both  symmetric  and  asymmetric  terrains). 

•  Need  to  research  split  mu  metrics  (gravel  shoulder  correction). 

•  Need  to  research  dynamic  stability  control  metrics. 

•  Improved  Tire/Track  to  soil  interface  force  predictions  (addresses  split  mu,  too) 

•  3D  dynamics  (also  includes  computer  control  ABS/ESC/active/semi-active,  etc.) 

•  Urban  maneuverability 
USA,  Karl  lagnemma 

•  Rigorous  representation  and  propagation  of  uncertainty  through  to  output  metric(s) 

•  Exploitation  of  modem  numerical  multibody  dynamic  modeling  methods  to  mitigate  reliance  on  ad  hoc 
correction  factors 

USA,  Mike  McCullough 

Incremental  evolutionary  approach  that  addresses  low-hanging  fruit  first 

•  closed  form  model  cleanup  and  expansion  (removes  some  parameter  redundancies,  expands  some  metrics) 

•  undercarriage  clearance, 

•  power  train  characteristics,  fuel  economy 

•  turning  performance 

•  vehicle  intrinsic  amphibious  characteristics  (i.e.  function  of  weight  and  CG  and  geometry  and  does 
not  require  dynamic  simulations  of  amphibious  operations) 

•  stationary,  ergodic,  spectrally  general  terrain  sample  definitions  for  ride  quality 

•  driver  feedback  loop  for  speed  control 

•  3D  Multibody  vehicle  dynamic  models  for  ride  quality,  including  driver  heading  control 

•  Deformable  terrain  in  terrain  and  mission  profile  definitions  (soil  type  and  moisture  content) 

•  3D  Multibody  vehicle  dynamic  models  for  obstacle  crossing  including  library  of  selectable  and  expandable 
standard  obstacles 

•  Add  dynamic  simulation  of  powertrain  performance  on  slopes  and  fuel  economy/range  with  3D  mission 
profiles  to  account  for  turning  effects  on  fuel  economy 

•  Expansion  of  obstacle  library 

•  rubble  pile  definition 

•  amphibious  operations  defined  by  dynamic  simulations 
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USA,  Jody  Priddy 


•  Complete  software  recoding  using  modern  programming  languages,  software  engineering  techniques, 
graphical  user  interfaces,  and  a  highly  modular  software  architecture. 

•  Software  licensing  that  imposes  minimal,  and  preferably  no,  upfront  purchasing  or  recurring 

•  Maintenance  costs  on  end-users  for  use  in  government  purposes. 

•  Software  license  rights  for  use  in  government  purposes  that  closely  result  in  “unlimited  rights”,  as  defined  in 
the  Defense  Federal  Acquisition  Regulation  Supplement  (DFARS)  of  the  U.S.  Department  of  Defense. 

•  New  powertrain  performance  modeling  capabilities  that  can  quantify  the  amount  of  driving  and  braking 
torque  that  will  be  applied  to  each  traction  element  of  ground  vehicles  with  conventional  powertrain 
architectures  during  mobility  operations  involving  a  comprehensive  array  of  vehicle  terrain  interaction 
scenarios,  which  should  include  powertrain  cooling  considerations. 

•  New  3-D  multibody  dynamics  M&S  capabilities  that  comprise  all  the  proven  capabilities  of  the  current  2-D 
vehicle  dynamics  preprocessor  VEHDYN  and  the  flexibility  to  address  numerous  ground  vehicle  mobility 
problems  well  beyond  the  scope  of  VEHDYN. 

•  New  capabilities  for  quantifying  the  influence  of  steering  system  performance  on  mobility. 

•  New  capabilities  for  predicting  other  mobility  performance  metrics,  with  particular  emphasis  on  including 
additional  output  metrics  desired  by  other  NATO  nations  in  addition  to  those  preferred  by  the  United  States. 

•  New  capability  to  select  from  and  use  multiple  analytical  terramechanics  modeling  alternatives,  based  on  the 
end-user’s  preference,  which  could  include  the  ability  to  “plug-in”  end-user  developed  terramechanics 
algorithms. 

•  New  terrain  characterization  and  terrain-state  forecasting  capabilities  for  producing  theater  specific  data  sets 
in  less  time,  with  higher  resolution  and  accuracy,  and  accounting  for  a  broader  array  of  terrain  features,  to 
include  urban  features. 

•  New  capabilities  to  account  for  the  influence  of  urban  features  on  mobility  performance  of  ground  vehicles 
(e.g.,  constricted  areas  due  to  high  urban  traffic  and  clutter,  tight  intersections,  narrow  roads,  etc.). 

•  New  capabilities  to  appropriately  account  for  the  influence  of  passive  and  active  control  systems  for  traction, 
suspension,  etc.  on  mobility  performance,  which  could  include  the  ability  to  “plugin”  secured,  proprietary, 
vendor-developed  controller-logic  modules. 

•  New  numerical  modeling  capabilities  for  terrain  physics  that  can  reduce  the  reliance  on  physical  testing  for 
terramechanics  relationships  while  providing  good  prediction  confidence  for  typical,  evolving,  and  atypical 
ground  vehicle  designs. 

•  New  powertrain  performance  modeling  capabilities  that  can  quantify  the  amount  of  driving  and  braking 
torque  that  will  be  applied  to  each  traction  element  of  ground  vehicles  with  hybrid  electric  and  fully  electric 
powertrain  architectures  during  mobility  operations  involving  a  comprehensive  array  of  vehicle-terrain 
interaction  scenarios,  which  should  include  powertrain  cooling  considerations. 

•  New  capabilities  to  address  mobility  performance  considerations  for  manned  and  unmanned  ground  vehicles 
that  require  quantified  influence  of  sensor,  perception,  and  autonomy  system  capabilities  on  mobility 
performance. 

•  Improved  human  response  M&S  capabilities  for  broader  quantification  of  human-specific  biophysical 
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limiters  on  mobility  performance  of  manned  ground  vehicles. 

•  Improved  M&S  capabilities  that  account  for  3-D  effects  during  fording  and  swimming  performance  in 
water-filled  linear-feature  gaps  and  coastal  features. 

USA  Brian  Wojtysiak 

•  Speedy  execution  (single  run  in  minutes  or  less,  not  hours  or  days) 

•  Ability  to  “play”  multiple  fidelity  levels  (e.g.  low  data  resolution  -  “red”  systems  /  paper  concepts  and  high 
fidelity  -  3D  modeling  and  vehicle  dynamic  behavior)  (fidelity  tradeoffs  are  sometime  necessary) 

•  Improve  user  interface  -  (e.g.  graphical  user  interface  (GUI)  for  inputs,  outputs,  and  data  management) 

•  Ability  to  build  new  and  /  or  update  existing  terrains  with  GIS  data 

•  Improve  NRMM  /  Geospatial  (ArcGIS)  interface  to  produce  cartographic  products 

•  Ability  to  verify  and  validate  model  predictions  with  vehicle  performance  data  (test  data) 

•  Update  NRMM  to  include  prediction  capabilities  for  light  weight  systems  (such  as  unmanned  ground 
vehicles,  robotic  systems) 

•  Eliminate  errors  in  statistical  output  generation  -  (e.g.  inaccessible  areas  surrounding  a  “Go”  area  -  ArcGIS 
mapping  software  can  be  used  to  eliminate  obvious  inaccessible  areas) 

•  Similar  metrics  for  measuring  how  "good"  a  vehicle  performs  (both  linear  and  areal). 

•  More  robust  reason  codes  and  options  for  diagnostics 

•  Allow  reporting  of  multiple  reasons  for  No-Go 

•  Be  able  to  easily  view  desired  calculated  variable  values  (e.g.  display  intermediate  prediction  results) 

•  Allow  for  hull  contact  with  surfaces  and  factor  in  the  associated  resistance  for  obstacle  performance 

•  Eliminate  issue  with  discrete  terrain  unit  transitions  -  step  function  differences  in  performance  at  NTU 
borders  -  results  should  “blend  /  transition”  at  the  NTU  boundaries 
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Appendix  D  -  THEME  2,  NORMMS  DETAILED  METHODOLOGY 

Michael  McCullough 


The  NG-NRMM  requirements  described  in  Chapter  6  and  Appendix  C  are  a  broad  list  of  capabilities  that  can 
be  broken  down  into  two  broad  categories:  mobility  modeling  process  improvements  and  mobility  metric  (i.e., 
“product”)  improvements.  “Process”  refers  to  how  mobility  models  should  be  implemented  to  promote 
commonality  and  standardization  as  well  as  ease  of  use,  etc.  These  requirements  and  recommendations  refer 
to  the  latest  modeling  methods  tools,  templates,  data  and  computational  capabilities  that  are  now  commonly 
available,  but  which  the  current  NRMM  is  not  able  to  leverage  to  advantage.  “Product,”  in  the  context  of 
mobility  models,  refers  very  specifically  to  new  or  updated  mobility  metrics,  including  adoption  of  specific 
algorithms  and  standards. 

The  NORMMS  address  both  process  and  product  improvements  for  the  NG-NRMM  and  can  be  developed  in 
a  top  down,  incremental  spiral  approach  with  progressively  higher  levels  of  resolution  developed  in  each 
iteration.  The  NORMMS  development  process  also  provides  high  level  “buckets”  into  which  the  early 
“ground  level”  contributions  and  issues  associated  with  very  specific  improvements  to  existing  metrics  are 
captured.  Eventually  the  top-down  spiral  development  process  will  progress  to  the  lowest  level  and  each  of 
these  early  detailed  specifications  will  be  already  complete  and  ready  for  inclusion  in  the  standard.  This 
approach  also  promotes  collaborative  parallel  development  as  each  member  of  the  RTG  can  work  the  issues 
unique  to  their  expertise  and  concern. 

Some  specific  examples  of  “ground  level”  improvements  that  are  already  being  proposed  are  provided  below. 


1 .  Ride  Quality.  Rainer  Gericke  proposed  that  the  NG-NRMM  should  expand  the  available  ride  quality 
metrics  to  include  ISO  2631-5  using  3D  metrics  applied  to  results  from  vehicle  testing  or  3D 
multibody  dynamics  models  with  embedded  high  resolution  tire  and  track  models.  He  also  proposed 
that  road  and  terrain  roughness  measures  be  defined  and  reported  consistent  with  ISO  8608. 

Consistent  with  this  proposal  and  the  need  to  maintain  the  historical  databases,  this  draft  ground  level 
specification  is  written  to  include  both  the  existing  metrics  and  the  proposed  new  metrics.  Rainer  has 
also  offered  a  validation  data  set  and  some  code  that  implements  some  of  these  calculations. 

a.  NG-NRMM  Threshold:  Driver’s  Vertical  Ride  Quality  shall  be  computed  as  6  watt  absorbed 
power  ride  limiting  speeds  versus  terrain  RMS  elevation  roughness  for  vertical  acceleration 
motion  inputs  at  the  occupant  seat  pans  where  the  absorbed  power  transfer  function  from 
Pradko  (1966)  is  applicable  and  the  terrain  RMS  elevation  roughness  is  measured  for  a  de¬ 
trended  terrain  profile  using  an  exponentially  weighted  de -trending  filter  with  lambda  =  10  ft, 
per  Murphy  (1984).  The  vertical  acceleration  data  must  be  generated  from  test  or  a  verified 
and  validated  vehicle  dynamics  model.  Ride  quality  can  be  additionally  computed  and 
specified  using  the  metrics  specified  in  ISO  2631-5.  Terrain  roughness  can  be  additionally 
described  and  reported  per  ISO  8608. 
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i.  Verification/Validation  Basis:  Current  VEHDYN  4.3  supplied  data  and  examples 

b.  NG-NRMM  Objective:  Driver’s  ride  quality  limits  shall  be  computed  in  all  three  orthogonal 
directions  with  the  following  respective  ride  limiting  speeds:  6  watts  vertical,  X  watts 
longitudinal,  Y  watts  lateral.  These  must  be  based  upon  acceleration  motion  inputs  at  tbe 
occupant  seat  pans  where  the  absorbed  power  transfer  functions  from  Pradko  (1966)  are 
applicable  and  tbe  terrain  RMS  elevation  roughness  is  measured  for  a  de -trended  terrain 
profile  using  an  exponentially  weighted  de -trending  filter  with  lambda  =  10  ft  per  Murphy 
(1984).  These  data  must  be  generated  from  test  or  a  verified  and  validated  vehicle  dynamics 
model.  Ride  quality  shall  be  additionally  computed  and  specified  using  the  metrics  specified 
in  IS02631-5.  Terrain  roughness  shall  be  additionally  described  and  reported  per  ISO  8608. 

c.  Verification/Validation  Basis:  Public  domain  data  set  on  a  standard  vehicle  (e.g., 
HMMWV) 

2.  Trafficability.  Dr.  J.  Y.  Wong  has  submitted  a  formal  proposal  for  a  module  to  compute  off -road 
traction  and  speed-made-good  using  a  steady  state  force  balance  based  on  the  application  of  Terra- 
mechanics  and  actual  Bevameter  measurements.  It  addresses  the  threshold  NG-NRMM  requirements 
by  focusing  on  conventional  manned  wheeled  and  tracked  vehicles  using  physics  basis  at  a  level  of 
geometric  resolution  appropriate  for  tire  and  track  interaction  with  terrain  (i.e.,  Bekker-Wong-Janosi 
basis  for  terrain  strength  modeling),  while  accounting  for  grades,  soils,  moisture  content,  and  snow. 
Extension  to  3D  by  directly  embedding  the  vehicle  terrain  interaction  computation  of  this  module  into 
a  multibody  dynamics  code,  allows  it  to  address  autonomous  vehicles  and  the  broader  range  of  3D 
metrics  to  include  turning,  fuel  economy,  integration  with  flexible  bodies,  vehicle  powertrain,  and 
steering  and  control  systems.  Dr.  Wong  also  summarized  the  available  documentation  and  approach 
to  leveraging  benchmarks  examples  for  validation  and  a  realistic  path  to  accumulation  of  vehicle 
terrain  interaction  (VTI)  data  for  future  validation.  Trafficability  has  traditionally  been  computed 
using  lower  resolution  whole-vehicle  empirical  metrics  such  as  Vehicle  Cone  Index  (VCI)  and  mean 
maximum  pressure  (MMP).  Those  legacy  approaches  have  been  widely  used  in  their  respective 
countries  of  origin  and  represent  valuable  legacy  metrics  with  large  legacy  databases.  The  latest 
version  of  NRMM  (version  2.8.2)  and  its  associated  vehicle  dynamics  program,  VEHDYN4.3, 
implement  rating  cone  index  (RCI)  based  pressure  sinkage  relationships  that  attempt  to  move 
incrementally  in  the  direction  of  a  more  semi -empirical  approach  envisioned  for  the  NG-NRMM. 
Therefore  the  following  draft  NORMMS  are  proposed  to  facilitate  an  orderly  transition  away  from  the 
purely  empirical  approach: 

a.  NG-NRMM  Threshold:  Trafficability  maps  must  be  based  on  validated  VTI  models  that 
utilize  soil  properties  that  are  available  from  validated  remote  sensing  methods.  Use  of 
vehicle  cone  index  (VCI)  values  that  have  been  demonstrated  via  test  with  a  real  vehicle  are 
acceptable  where  necessary,  but  users  should  be  forewarned  that  VCI  has  demonstrated 
limitations  and  will  eventually  be  superseded  by  formulations  implementing  terramechanics 
and  continuum  mechanics  models  of  VTI  which  have  the  potential  to  enable  eventual 
utilization  of  remote  sensing  data  for  soil  characterization  and  calculation  of  trafficability  at 
the  tire  and  track  block  level  of  resolution. 


UNCLASSIEIED:  Distribution  Statement  A.  Approved  for  public  release;  distribution  is  unlimited 


173 


i.  Verification  Basis:  Current  NRMM  v2.8.2b  supplied  data  and  examples  and  any 
additional  VTI  data  supporting  Bekker-Wong-Janosi  (or  equivalent)  models  at  the  tire 
and  track  block  level  of  resolution. 

b.  NG-NRMM  Objective:  Trafficability  maps  and  models  must  be  based  on  validated  VTI 
models  at  the  tire  and  track  block  level  of  resolution  and  below  (continuum  models),  that 
utilize  soil  properties  that  are  valid  for  extrapolation  to  terrains  for  which  the  only  data 
available  are  from  remote  sensing  methods. 

i.  Verification  Basis:  to  be  developed  (TBD) 

3.  Real  time  mobility  model  metrics.  Dr.  Vladimir  Vantsevich  has  suggested  that  many  of  these  metrics 
may  find  useful  application  in  real  time  control  of  vehicle  systems  and  therefore  their  efficient 
formulation  for  these  purposes  might  become  an  important  branch  of  the  NG-NRMM  effort. 


Murphy,  N.  R.  Jr.,  1984.  A  Method  for  Determining  Terrain  Surface  Roughness,  US  Army  Waterways 
Experimentation  Station,  Geotechnical  Laboratory,  Vicksburg  MS,  Sept  1984. 

Pradko,  F.,  R.  Lee  and  V.  Kaluza,  V.  1966.  Theory  of  Human  Vibration  Response,  presented  at  the  Winter 
Annual  Meeting  and  Energy  Systems  Exposition  of  the  American  Society  of  Mechanical  Engineers,  New 
York. 
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Appendix  E  -  REQUEST  FOR  INFORMATION  (THEME  5) 

Henry  Hodges 


E.l  LETTER  INTRODUCING  REQUEST  FOR  INFORMATION 


Nevada  Automotive  Test  Center 


A  Division  of  Hodges  Transportation,  lr>c. 

Real  Time, 

Real  World 
Solutiofts~^^^ 


P.O.  Box  234 
Carson  City,  Nevada  89702*0234 
Phone:  (775)629*2000 
Fax:  (775)629*2029 
email:  hhodgesjr@natc*ht.com 


17  March  2015 

Mr.  XXX 
Company 
Address 

Email: 

Subject:  Request  for  Information  on  Tools  which  can  provide  a  Ground  Vehicle  Mobility  Simulation 

Environment 


Dear 

NATO  Applied  Vehicle  Technology  (AVT)  Panel  has  established  an  Exploratory  Team  (ET)  to  potentially 
identify  and  recommend  physics  based  simulation  tools  which  can  be  used  to  substantially  improve  the 
eapabilities  of  the  existing  NATO  Reference  Mobility  Model  (NRMM).  Your  organization  has  been 
identified  as  having  developed  simulation  tools  which  could  be  used  to  substantially  improve  the  Modeling 
and  Simulation  environment  neeessary  to  aceurately  prediet  vehicle  performanee  in  both  established  and 
marginal  terrain  eonditions.  The  attaehed  document  explains  the  type  of  information  required  to  support  this 
evaluation  effort  and  identifies  the  criteria  to  be  used. 

Please  provide  your  information  and  questions  regarding  this  effort  to  the  ET  Theme  5:  Tool  Choiees  Lead 
identified  below. 

Henry  Hodges 
President 

Nevada  Automotive  Test  Center 
PO  Box  234 

Carson  City,  Nevada  89702 
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USA 


hhodgesir@natc-ht.com 

Phone:  775-629-2000 

The  process  for  evaluation  is  expected  to  he  similar  to  that  used  for  the  United  States  Marine  Corps 
Simulation  Based  Acquisition  effort  utilized  during  the  Logistics  Vehicle  System  Replacement  program.  As 
such  the  information  provided  will  he  reviewed  as  appropriate  hy  the  NATO  ET-148  committee  and  more 
specifically  hy  technical  representatives  of  the  US  Army  TARDEC  and  US  Marine  Corps  Systems  Command. 
Solutions  which  are  capable  of  providing  and  supporting  the  future  mobility  systems  analysis  architecture  for 
wheeled  and  tracked  vehicles  including  autonomous  vehicle  systems  will  be  identified. 

The  efforts  of  the  NATO  ET-148  Committee  will  be  published  and  that  information  provided  to  the 
appropriate  Governmental  and  Commercial  user  communities. 

Your  response  must  be  provided  not  later  than  16  March  2015  in  order  to  support  the  full  ET  meeting  and 
review  scheduled  for  the  week  of  20  April.  Early  submittal  of  the  information  will  allow  time  for  discussions 
to  insure  that  your  approach  is  clearly  understood.  Additional  questions  will  be  provided  as  necessary. 

Should  you  have  any  questions  please  contact  Henry  Hodges  as  identified  above  or  Dr.  Paramsothy 
Jayakumar  (paramsothy .j  ayakumar.civ  @  mail.mil) . 


Respectfully, 


Henry  Hodges 
President 

NEVADA  AUTOMOTIVE  TEST  CENTER 
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E.2  INTRODUCTION 


Ground  vehicles  are  deployed  worldwide  in  many  challenging  environments.  Whether  tracked  or  wheeled, 
the  challenges  for  successful  and  safe  operation  continue  to  increase  due  to  environmental  extremes  and 
regional  instahilities.  Over  the  past  20  years  ground  vehicle  technology  has  vastly  improved,  allowing 
vehicles  to  successfully  operate  over  rugged  terrain.  However,  often  times  the  design  and  production  of  those 
vehicles  is  generated  thousands  of  miles  from  where  those  vehicles  operate.  The  ability  of  the  vehicles  to 
successfully  complete  a  humanitarian  or  operational  mission  cannot  he  determined  until  the  vehicles  are  in  the 
field  and  this  creates  significant  risk  to  all  involved.  Through  satellite  and  other  data  collection  methods,  the 
ability  to  identify  terrain  conditions  in  terms  of  vegetation,  slope,  obstacles,  and  environmental  extremes  due 
to  excessive  rain  or  drought  has  approached  near  real  time  information.  Therefore,  it  is  appropriate  to 
consider  a  physics  based  simulation  environment  which  can  assess  and  predict  the  performance  of  wheeled 
and  tracked  vehicles  in  these  operating  conditions.  Such  a  simulation  environment  would  allow  not  only  the 
accurate  development  of  a  successfully  mobile  and  reliable  vehicle  but  also  a  predictive  tool  to  determine  the 
applicability  of  that  vehicle  to  current  operational  requirements.  It  is  also  recognized  that  the  availability  of 
high  performance  computing  is  further  enabling  cost  and  time  effective  detailed  modeling  of  the  vehicle 
terrain  system  providing  high  fidelity  simulations. 

The  purpose  of  this  Request  for  Information  is  to  determine  the  availability  of  such  tools  and  to  establish  a 
sustainable  simulation  environment  which  has  the  flexibility  to  incorporate  new  simulation  solutions  as  they 
are  developed.  It  is  further  noted  that  continuing  and  new  research  development  are  necessary  in  specific 
technology  areas.  As  such  a  “template”  based  simulation  environment  is  envisioned  under  the  following 
charter  The  framework  is  a  ground  vehicle  mobility  modeling  and  simulation  architectural  specification 
applicable  to  the  full  range  of  ground  vehicle  geometric  scales  that  promotes  standardization, 
integration,  modular  interoperability,  portability,  expansion,  verification  and  validation  of  vehicle- 
terrain  interaction  models  at  multiple  levels  of  theoretical  and  numerical  resolution. 

Physics-based  simulation  environments  are  currently  available  either  commercially,  open  source, 
academically,  or  within  Government  agencies.  New  simulation  environments  are  being  developed 
specifically  to  support  current  challenges  from  man-machine  interface  to  complete  vehicle  autonomy.  The 
vision  of  the  RFI  is  to  collect  available  information  for  the  physics-based  vehicle  and  the  environment  in 
which  that  vehicle  operates  to  utilize  that  information  to  establish  the  criteria  for  the  framework  and  to 
conduct  a  downselect  with  the  outcome  being  a  recommendation  for  a  successful  framework  which  would  be 
available  for  implementation  throughout  the  NATO  member  countries  within  three  years. 

This  RFI  seeks  information  specific  to  ground  vehicle  dynamics  simulation,  terrain  mapping  and  autonomy 
capabilities.  A  separate  questionnaire  for  each  of  these  is  provided  in  the  attachments. 
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E.3  HISTORY 


Empirically  based  tools  such  as  the  current  NATO  Reference  Mobility  Model  (NRMM)  have  been  used  to 
compare  and  generally  rank  order  the  mobility  capability  of  tracked  and  wheeled  vehicle  systems.  This  tool 
was  originally  developed  during  the  1970  time  frame  and  has  been  updated  several  times  since  then.  While 
this  tool  has  generally  successfully  served  its  purpose,  current  technology,  both  in  terms  of  computing  speed 
and  physics  based  simulation,  can  now  potentially  provide  a  significant  improvement  both  in  terms  of 
accuracy  and  the  ability  to  predict  vehicle  performance  in  near  current  conditions  and  for  both  traditional  and 
future  concept  vehicle  configurations. 

Many  tests  and  evaluations  have  been  performed  utilizing  the  principle  tenets  of  the  NRMM.  It  is  appropriate 
to  build  on  those  lessons  learned  and  therefore  take  advantage  of  the  capabilities  established  from  NRMM. 
These  capabilities  have  included:  The  ability  to  compute  tactical  mobility  metrics  by  integrating  engineering 
level  performance  capabilities  onto  different  terrain  conditions.  This  approach  allowed  successful  comparison 
of  various  vehicle  systems  and  capabilities  over  varied  terrain  surfaces,  obstacles,  vegetation,  weather 
scenarios,  grades  and  other  features  which  can  adversely  impact  vehicle  performance. 

The  NRMM  was  provided  without  charge  to  approved  end  users.  Attachment  2  provides  a  list  of  the 
typical  data  input  requirements  for  NRMM.  This  data  is  expected  to  be  a  subset  of  the  requirements 
for  a  more  advanced  simulation  environment. 

Features  included: 

•  Quick  run  times  allowing  studies  involving  multiple  vehicles  to  be  completed  in  relatively 
short  time  frames 

•  The  ability  to  assess  the  combined  effects  of  a  variety  of  off-road  challenges  (soil  strength, 
grades,  obstacles,  vegetation,  ride  and  shock  tolerances,  weather  conditions,  human  factors, 
etc.) 

•  Diagnostic  reason  codes  to  help  understand  results 

•  Empirical  relations  (i.e.,  VCI  vs.  drawbar/resistance)  that  provide  a  level  of  self-validation 

•  The  ability  to  conduct  evaluation  of  both  wheeled  and  tracked  vehicles  over  similar  terrain 
conditions 

•  The  ability  to  rapidly  evaluate  the  effects  of  sub-system  design 

•  Outputs  which  can  be  represented  with  maps  (speed  maps,  speed  comparison  maps)  for  better 
visualization/comparison  (if  digital  terrain  file  available). 
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E.4  GROUND  VEHICLE  MOBILITY  SIMULATION  ENVIRONMENT 


It  is  intended  that  the  next  generation  analysis  tool  would  retain  the  positive  attributes  of  NRMM  while 
overcoming  a  number  of  limitations  identified  which  have  adversely  impacted  the  ability  to  quantify  the 
performance  of  vehicle  systems  which  utilize  technologies  not  previously  been  incorporated  into  the  empirical 
nature  of  the  tool. 

Some  of  the  goals  for  the  next  generation  physics  based  modeling  and  simulation  tool  include: 

1.  The  ability  to  evaluate  ride  quality  and  mobility  of  the  vehicle  over  a  three  dimensional  terrain 
environment  which  would  include  the  following 

a.  3D  vehicle  multi-body  dynamics  models  that  are  more  precisely  representative  of  vehicle 
designs,  including  flexible/deformable  bodies,  stabilization  and  control  system  hardware  and 
software,  etc. 

b.  Multiple  deformable  terrain  surface  types  within  the  simulation,  including  soil,  snow,  ice, 
freezing/thawing  ground,  vegetation  effects,  etc. 

c.  Terrain  specifications  for  mission  profiles 

i.  Spectral  content  of  the  elevation  geometry  &  roughness 

ii.  Variable  soil  and  vegetation  type 

iii.  Ergodic  and  stationary  geometric  sample  lengths  with  respect  to  ride  quality  response 
parameters  (accounting  for  skid  plate,  drive  sprocket  or  idler  contact  events) 

d.  Driver  feedback  loop  model  for  speed  and  direction  control  of  3D  vehicle  dynamic  model, 
including  drivers  with  different  levels  of  experience  (beginner,  novice  and  advanced) 

e.  Automated  iterative  loops  for  determining  the  speed  limits  to  obtain  6  watts  of  absorbed 
power  and  2.5  g  vertical  response  at  occupant  locations,  or  similar  metrics  as  specified. 

i.  Access  to  intermediate  and  lower  level  results  plots  such  as  speed  vs  power,  acceleration 

2.  Improved  obstacle  crossing  metrics  which  include  for  example 

a.  3D  Deformable  terrain  with  embedded  hard  obstacles  in  the  simulations 

b.  Rubble  pile  definition  and  standardization 
i.  Could  include  dynamic  rubble 

c.  Library  of  obstacles  that  are  selectable  and  tailorable  to  vehicle  and  mission  requirements 

d.  Amphibious  operations  obstacles 

i.  Stream/lake  fording 

ii.  Surf  zones  including  rocky  shores 

iii.  Ship  launch 

3.  Off-road  mobility 

a.  Prediction  of  tire  and  track  sinkage  in  various  soil  conditions 

b.  Prediction  of  vehicle  ability  to  negotiate  dry  and  wet  soil  slopes 

c.  Prediction  of  vehicle  maneuverability  while  turning  in  soft  soil  conditions 

d.  Ability  to  load  current  or  near  real  time  terrain  information  to  establish  optimum  travel  path 
based  on  vehicle  capabilities  and  environmental  conditions 

e.  Stability  while  negotiating  severe  terrains  on  various  slopes  while  avoiding  obstacles 

f.  Predicted  fuel  economy  during  mobility  operations 
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4.  On  road  performance 

a.  Prediction  of  speed  on  grade 

b.  Analysis  of  vehicle  during  dynamic  maneuvers  including  obstacle  avoidance,  severe  lane 
change,  moose  avoidance,  road  departure  recovery 

c.  Analysis  of  run  flat  and  variable  tire  pressure  on  vehicle  stability,  understeer/oversteer 
characteristics,  driver  in  the  loop 

5.  Autonomous  (Intelligent)  vehicle  mobility 

a.  Integration  of  control  algorithms  for  all  drive  by  wire  functions 

b.  Optimization  of  control  functions  for  terrain  and  operational  requirements 

c.  Ability  to  provide  real  time  feedback  from  vision,  LIDAR  and  vehicle  sensor  arrays 

d.  The  autonomous  vehicle  mobility  challenges  are  increased  due  to  the  requirement  to  stop- 
sense-  determine  -  proceed  functionality.  This  places  higher  demands  on  the  soft  soil 
mobility  prediction  capability  due  to  the  increased  torque  and  braking  impulse  loads  and  the 
fact  that  the  system  can  no  longer  rely  on  inertia  to  negotiate  short  duration  high  mobility 
demand  events. 

6.  Improved  powertrain  representation  which  reflects  digitally  controlled  engine,  transmission, 
transfer  case,  differentials,  geared  reduction  hubs,  hybrid  electric  technology,  etc.,  which  allows 
accurate  performance  prediction  for  soft  soil  slopes  and  fuel  economy/range  prediction  over 
terrain  which  produces  variable  motion  resistance  conditions 

7.  Improved  uncertainty  analysis  as  a  function  of  vehicle  and  terrain  variability  or  available  data 
precision/imprecision 

8.  Simulation  capability  to  run  on  various  platform  from  desktop  to  HPC 

a.  In  order  to  meet  the  objective  to  rapidly  provide  comparative  results  it  is  expected  that  a 
version  of  the  next  generation  mobility  simulation  will  function  capably  in  a  desktop  parallel 
processor  based  platform.  A  more  robust  and  detailed  version  which  would  retain  fidelity  of 
soil  conditions  through  the  thermal  degradation  of  shock  absorbers  would  then  function 
successfully  in  a  much  higher  speed  processing  environment 

b.  Within  the  simulation  environment,  evaluation  of  hardware  in  the  loop  is  expected.  As  noted 
later  in  this  document,  dynamic  analysis  including  control  feedback  loops  at  relatively  high 
update  rates  are  required  to  reflect  current  vehicle  technologies 

Table  1  generally  describes  the  vision  of  how  the  modeling  approach  will  progress  from  the  current 
empirically  based  environment  to  a  full  physics  based  simulation  environment.  Throughout  this 
process  lessons  will  be  learned  to  identify  the  critical  elements  for  successful  prediction  of  manned 
and  unmanned  systems. 
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Table  E.l.  Next  Generation  NRMM  Methodology  Classifications 


Model 

Component 
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Empirical - 
Current 

Empirical - 
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Off-Road 

Mobility 
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o 
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^  _ 

c  ^ 

^  § 
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•5-  p 

</) 

■55 

55  1 

Intelligent  S  Not  1 

Vehicle  |  Available  i  Autonomy 

z  z 

1  1 

c  2 

Autonomy  < Autonomy 

i  ^ 

n  s 

Compute 

Platform 

Desktop 

1  Desktop 

w 

Desktop 

TS 

S  HPC 

1 

Attachment  3  provides  a  list  of  model  data  requirements  which  could  he  expected  in  an  advanced  vehicle 
simulation  to  achieve  these  goals. 

E.5  SIMULATION  STRUCTURE 

As  noted  above  the  intent  of  the  effort  is  to  develop  a  structure  which  allows  current  and  future  tools  to  he 
introduced  in  a  core  simulation  environment.  An  open  architecture  structure  is  anticipated  which  will  allow 
specifically  developed  tools  to  support  improvement  of  simulation  fidelity.  A  significant  level  of  effort 
involved  in  physics  based  simulation  is  the  development,  input  and  connection  of  vehicle  component 
parameters  to  successfully  represent  the  entire  vehicle  system.  Detailed  simulations  can  be  developed  which 
range  from  analysis  of  the  combustion  dynamics  of  an  engine  to  driver -in-the -loop/cognitive  recognition 
estimations.  When  predicting  or  comparing  vehicle  capabilities  and  performance  over  different  mission 
events,  the  level  of  fidelity  of  certain  components  or  capability  may  be  more  important  for  certain  vehicle 
aspects  than  for  others.  The  intent  of  the  effort  is  to  create  a  simulation  environment  which  will  allow  the 
level  of  fidelity  or  precision  for  various  components  or  systems  to  be  varied  from  simple  to  complex  to  aid  in 
the  speed  of  the  analysis.  For  example,  retaining  non-linear  bushing  attributes  while  determining  a  300-mile 
mission  profile  fuel  economy  comparison  is  not  necessary.  However,  when  predicting  accurate  soft  soil 
mobility,  retention  of  precise  dynamic  tire  footprint  force,  shear  and  pressure  parameters  along  with  soil 
reaction  may  be  critical.  Regardless  of  simulation  intent,  the  environment  should  allow  data  to  be  drawn  from 
a  common  vehicle  system  data  set  as  appropriate  for  the  intent  of  the  simulation.  A  description  of  this 
capability  is  requested  as  part  of  the  response  to  this  RFI. 

The  physics  based  environment  should  successfully  provide 

•  Vehicle  based  GUI  instead  of  generic  modeling  and  simulation  interface 
o  Automatic  left/right  symmetry  where  appropriate 

o  Vehicle  terminology  and  correlation  to  Bill  of  Materials  for  the  configuration 
o  Include  custom  vehicle  simulation  events 
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o  Include  vehicle  specific  post  processing 

•  API  to  extend  the  system  to  meet  future  demands 

•  Utilities  to  support  unique  modeling  elements,  such  as  tire  models. 

•  Library  of  vehicle  templates 

o  Build  on  previously  established  and  validated  vehicle  simulations 
o  Evaluate  alternative  suspension,  drive  train,  stability  control  systems 
o  Provide  access  to  existing  component  data  (tires,  bushings,  springs,  etc.) 
o  Provide  access  to  existing  terrain  and  soil  data 

•  Standard  modeling  practices 

•  Database  hierarchy  for  storing  all  data 

•  Standardized  format 

•  Interface  with  various  FEA  simulation  tools  for  flexible  bodies,  and  automatic  stress  and  fatigue 
calculation.  Embedded  FEA  technology  could  be  a  plus 

•  Interface  with  various  controls  simulations  or  embedded  controls  functionality  with  a  sufficient 
library  to  satisfy  the  modeling  of  modern  controls  system,  now  and  in  the  future 

•  Ability  to  incorporate  hydraulic  systems 

•  Interface  with  man-  and  hardware-in-the-loop  (MIL  and  HIL)  simulations 

•  Evaluation  of  suspension  characteristics  before  integrating  with  full  vehicle 

•  Tire/Track/Soil  system  models 
o  Off-Road  with  3D  terrain 
o  Deformable  tire/terrain 

•  Mechanical  Subsystems  fully  represented: 

o  Suspension  (for  wheeled  and  tracked  vehicles) 
o  Powertrain 
o  Tires  (including  runflats) 

o  Tracks  (including  dynamic  track  tension  adjustment) 
o  Structure 
o  Steering 
o  Brakes 

•  Native  ability  to  support  design-of-experiments,  stochastic  studies  (e.g.  Monte  Carlo),  design  studies 
and  optimization 

•  Utilization  of  parallel  processing  or  other  demonstrated  techniques  to  yield  world-class  model 
execution  times.  This  includes  the  support  of  cloud  computing  on  common  cloud  HPC  (high 
performance  computing)  platforms 


When  implemented  the  simulation  environment  would  provide  capabilities  including: 

•  The  ability  to  validate  vehicle  dynamics  and  terrain  interaction  templates  through  physical  test 

•  The  ability  to  evaluate  vehicle  system  performance  against  events  which  are  representative  of  the 
operating  environment 

•  Prediction  of  vehicle  durability  and  impact  of  design  on  life  cycle  cost  through  fatigue  damage 
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analysis 

•  Analysis  of  system  performance  including  impact  of  system  degradation  on  vehicle  capability  and 
safety 

•  Simulation  Based  Acquisition  tools  which  can  be  used  to  support  selection  of  vehicle  systems  and 
components  for  vehicle  improvement 

•  Integration  of  electronic  controls 

•  Improved  tire  and  track  dynamics  models  capable  of  implementation  on  deformable  terrain 
o  Low  fidelity  and  high  fidelity  options 

•  Improved  deformable  terrain  models  capability  of  representing  a  broad  range  of  terrain  and 
environmental  conditions  (different  soils,  soil  strength  and/or  moisture,  variable  snow  conditions,  ice, 
freeze/thaw  layering) 

o  Low  fidelity  and  high  fidelity  options 

•  Saved  and  geospatially  referenced  terrain  deformation  information  (such  as  rutting). 

•  Mobility  predictions  on  deformable  soils  including  the  ability  to  traverse  level,  rough,  and  variable 
slope  terrain 


E.6  COMBINATORIAL  TRADE  STUDY 

The  information  provided  in  response  to  the  RFI  will  be  evaluated  using  various  criteria  ranging  from  the 
fidelity  of  the  simulation  environment  to  the  operating  cost  of  the  environment  to  the  ability  to  validate  the 
simulations  against  controlled  test  events  which  match  the  simulation  environment.  While  low  cost  is  an 
important  parameter,  the  fidelity  of  the  simulation  and  the  ability  to  validate  the  results  of  the  simulation  are 
very  important,  as  is  the  ability  to  perform  simulations  quickly.  To  address  these  conflicting  requirements,  a 
combinatorial  trade  study  (CTS)  analysis  will  be  conducted  which  utilizes  measures  of  performance  (MOP) 
and  measures  of  effectiveness  (MOE).  Currently  the  following  criteria  is  anticipated  in  broad  terms.  This 
CTS  criteria  approach  is  intended  to  aid  your  understanding  of  the  need  for  the  effort,  and  identifies  the 
priority  placed  on  the  various  elements  associated  with  the  simulation  environment.  It  is  expected  that  within 
your  RFI  response  that  each  of  the  elements  would  be  addressed.  Based  on  the  range  of  responses  received, 
the  CTS  will  be  updated  to  best  reflect  those  elements  which  will  ensure  the  most  flexible  and  accurate 
solution  for  next  generation  mobility  simulations. 
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Table  E.2.  Toolset  Scoring  Matrix 


MOP 

MOE 

importance 

Physics  Based 

11.25% 

Accuracy 

Vaiidation  through  measurement 

37.50% 

15.00% 

Supports  time  and  frequency 
domain  analysis 

11.25% 

Template  based 

8.65% 

Flexibility 

wheeled  or  tracked  vehicles 

37.50% 

14.42% 

Automotive  Subsystems 

14.42% 

License 

4.17% 

Cost 

Run  time 

12.50% 

5.56% 

Training 

2.78% 

NATO  specific 
applications 

Supports  unique  terrain  or 
mission  definition 

6.94% 

Worldwide  tooi  availability  to 
approved  sources 

12.50% 

4.17% 

world  wide  tool  support 

1.39% 

100.00%  I 

E.7  USER  ENVIRONMENT  AND  SUPPORT 

The  simulation  environment  will  support  both  occasional  and  expert  user  capabilities  and  that  online  training 
as  well  as  consulting  services  capability  would  be  available.  As  part  of  your  response  please  explain  the 
capability  of  your  simulation  environment  to  provide  a  controlled  user  environment  with  appropriate  graphical 
user  interface  (GUI)  as  well  as  an  expert  user  environment  where  new  capabilities  can  be  developed  and 
supported.  The  expert  user  should  be  provided  a  robust  API  to  allow  easy  creation  of  new  functionality.  Use 
of  common  languages,  such  as  Python,  is  a  plus.  As  part  of  support,  identify  the  market  penetration  of  your 
solution  as  well  as  the  presence  of  user  groups  and  consulting  support. 

The  template  style  environment  will  be  developed  to  aid  in  the  speed  and  fidelity  of  the  simulation 
environment.  As  such,  once  a  complete  vehicle  model  is  developed,  it  is  anticipated  that  components  and 
subsystems  can  be  rapidly  changed  and  the  simulation  rerun  without  the  need  to  completely  rebuild  the 
simulation.  For  example,  it  is  anticipated  that  the  suspension  system  envelope  would  be  defined  in  the  base 
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L#wtf 

4im 


model  and  that  geometrically  similar  passive  spring  and  dampers,  or  semi  active  struts,  or  adaptive 
suspension,  or  fully  active  suspensions  could  be  implemented  within  that  simulation  envelope  and  the 
simulation  rerun  to  quickly  contrast  and  compare  the  impact  on  the  overall  system  performance.  As  such  the 
suspension  might  be  represented  in  the  simulation  as  shown  below. 


Tm  rod  for  ti^tring 


$«npto  gfomotiy  drtvin 
by  hard  powit  alawi 
rap«d  deiigii 
mluatKMit 


Y4hap«d  upp«r 
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for  tpiino  rnlil 


Activt 


Modcltd  as  a  e&nirol 
tyilen  InMatFab 


ConlrOl  Arm 
eiffhlikgi 


Figure  El.  Suspension  envelope  created  using  templates  and  GUI 

An  input  table  as  part  of  an  existing  GUI  would  be  able  to  accept  various  vehicle  components  and 
configurations  and  would  include  both  flexible  and  rigid  components.  As  noted  below,  the  vehicle  system 
would  then  be  assembled  and  evaluated  over  representative  terrain  conditions  producing  predicted  results 
ranging  from  dynamic  stability  to  flexible  body  fatigue  analysis  to  deformable  terrain  tractive  effort. 
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Common  Input  Forms  for 
Engineering  Data 


Templates  Allows  Real  Vehicle 
Designs  to  be  Evaluated  in  a 
Common  Format 


Common 

Payloads  & 

J 

.•  .  -;A  ^  \  i 

CG 

- 

■'  V'-"  V  '"i 

Figure  E2 

Specific  performance  events  on  paved,  gravel  and  variable  surface  conditions  would  be  performed  and 
compared  directly  to  physical  test  events. 
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Figure  E3 
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I  22  Ton  Payload  Oshkosh  Constant  Radius  Payload  CG  134  in  above  ground  I 


IvsM  a_hiftTC_cr_cg_1  Time=  5.8000  Frame=122 


Figure  E4 


Figure  E5 
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Typical  List  of  Events 


•  Constant  Radius 

•  DouMe  Lane  Change  -  Paved  And  Gravel  Surface 

•  Road  Edge  Recovery 

•  30%  -  40%  Side  Slope  Slalom 

•  Mission  Profile  Trails  &  Cross-Counir, 

’  Washboard  Event 

•  30%  Dry  Sand  Grade 

•  40%  Dry  Sand  Grade 

•  Split-Coefficient  ABS  Stop 

•  24  -  36  Inch  Vertical  Step 

•  6  Inch.  8  Inch.  10  Inch  Half  Round 

•  Speed  on  5%  Paved  Grade 

•  Straight  Line  Acceleration 

•  Traditional  RMS  Course 

•  TradibonalWNS  Course 


Figure  E6 


E.8  CONTROL  ALGORITHMS 

Wheeled  and  tracked  vehicles  whether  equipped  with  traditional  powertrains,  hybrid  electric  or  other 
alternative  systems  are  digitally  controlled.  Therefore,  the  ability  of  the  simulation  environment  to  support 
accurate  representation  of  the  algorithms  which  control  the  interaction  of  the  components  is  essential  to 
accurate  results.  From  ABS  to  traction  control  to  stability  control  to  engine  and  transmission  systems, 
electronic  control  of  the  various  systems  dominates  the  performance  of  the  vehicle.  Please  explain  the  ability 
of  your  simulation  environment  to  accommodate  those  control  relationships  both  in  terms  of  software  and 
hardware  in  the  loop.  As  vehicle  systems  trend  toward  smart  or  autonomous  operation,  incorporation  of  on- 
vehicle  and  remote  sensing,  including  vision  based  systems  which  require  gigabit  rate  connectivity,  it  is 
necessary  to  accurately  represent  these  control  or  input  relationships  to  successfully  represent  the  vehicle 
system. 


E.9  VEHICLE-TERRAIN  INTERFACE 

In  an  off-road  environment,  the  tire  or  track  soil  interaction  is  critical  and  the  ability  to  accurately  represent 
that  envelope  is  vital  to  the  success  of  the  simulation.  The  intended  usage  for  a  deformable  soil  model  is  to 
evaluate  motion  resistance  (for  example  in  fuel  economy  simulations)  as  well  as  vehicle  tractive  effort 
capabilities  to  determine  trafficability.  The  models  should  be  able  to  differentiate  performance  when 
operating  on  different  types  of  soil  and  soil  conditions,  for  example  dry  coarse  grained  soil  versus  wet  fine 
grained  soil. 

In  additional  to  the  variety  of  soil  types  and  strengths  needed,  the  weather  effect  on  the  terrain  is  also  critical, 
thus,  the  capability  to  represent  soil  freeze/thaw  in  addition  to  snow  and  ice  conditions  are  critical  elements. 

It  is  recognized  that  there  are  many  approaches  to  soft  soil  modeling,  including  Bekker-Wong,  particle  based 
models,  finite  element,  boundary  element,  and  discrete  element  methodologies.  In  addition  to  soil 
deformation,  factors  such  as  tire  deformation,  footprint,  pressure  distribution,  and  tire  tread  pattern  can  all 
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significantly  impact  the  results.  Effeets  such  as  bulldozing,  and  the  sink/slip  relationship  for  the  tire  in 
deformable  soil  should  be  addressed.  The  response  to  this  RFI  should  clearly  define  the  approach  taken  for 
deformable  soil  modeling,  the  data  requirements,  and  the  model  capabilities. 

E.IO  TERRAIN  REPRESENTATION 

It  is  anticipated  that  within  the  advanced  simulation  environment  more  accurate  terrain  information  will  be 
made  available  and  the  vehiele  performance  over  that  terrain  sueeessfully  simulated. 

Terrain  Plot  for  Mission  Profile 


Soil 


Figure  E7 


Figure  E8  Roughness 
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Mission  Profile  Terrain 


Figure  ElO 


The  integrated  simulation  environment  would  allow  three  dimensional  operation  over  the  range  of  terrain 
conditions  as  descrihed  in  attachment  2  represented  in  the  following  definitions. 

The  terrain  elements  could  he  updated  for  current  conditions  resulting  from  environmental  changes  as  may 
occur  due  to  rain,  snow,  vegetation,  and  other  seasonal  events. 

Terrain  elements  will  he  given  values  according  to  a  terrain  code  using  algorithms  for  the  distribution  of 
vegetation  and  climate  conditions  including  rain  and  snow.  These  algorithms  will  he  derived  from  data  sets 
typically  associated  with  geographic  information  systems  (GIS).  The  data  will  he  used  to  construct  real  world 
based  simulation  in  the  modeling  environment  and  to  accurately  depict  this  environment  in  a  visual  format 
such  as  a  3D  map  where  possible.  Slope,  aspect,  and  soil  type  data  will  be  combined  with  the  climate 
condition  and  land  cover  data  and  include  such  variation  as  deciduous  versus  coniferous  trees,  tree  spacing, 
and  the  height  and  extent  of  forest  canopy,  all  of  which  have  a  direct  effect  on  the  watershed  of  rain  or  snow 
melt.  Combined  with  soil  strength  and  composition,  these  combined  elements  have  a  direct  impact  on  vehicle 
mobility.  These  terrain  and  climate  elements  are  essential  to  building  an  accurate  modeling  and  simulation 
environment  for  vehicle  mobility  predictability. 
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These  geographic  data  should  he  exportable  for  import  into  modeling  and  simulation  software  or,  if  already  in 
an  applicable  format,  exportable  to  Open  Geospatial  Consortium  (OGC)  formats  or  other  industry  standard 
file  types,  such  as  shapefiles,  for  inclusion  into  mapping  software. 

At  a  minimum,  these  elements  would  include  the  following  data  types  and  resolutions. 

•  Slope  in  the  form  of  a  digital  elevation  model  (DEM)  or  digital  terrain  model  (DTM)  with  0.5 

to  5  meter  resolution 

•  Land  cover/vegetation  data  at  1  to  30  meter  resolution 

•  Soil  information  consistent  with  NRCS  data  best  resolution  and  including  engineering  soil 

type  exportable  to  a  lookup  table.  This  should  also  reflect  rock  and  boulder  spacing  and  size 
as  well  as  vegetation  spacing 

•  Climate  data  by  month  (from  present  and  going  back  10  years)  at  10  meter  resolution 

minimum 

Import/export  capability  should  specifically  include  a  fully  3D  route  “swath”  either  as  designated  by  the  use 
or  automatically  generated  by  the  software. 

E.ll  RESPONSE 

The  above  information  and  the  following  attachments  are  intended  to  provide  background  and  guidance  in 
responding  to  the  questionnaires.  Responders  may  include  additional  information  which  will  be  considered. 
Product  information  videos  and  presentations  will  be  accepted  as  part  of  the  RFI. 

Attachment  1  -  Concept  mission  profile  database 

Attachment  2  -  Minimum  data  input  requirements 

Attachment  3  -  General  physics  based  model  data  input  requirements 

Attachemnt  4  -  Vehicle  dynamics  model  product  questionnaire 

Attachemnt  5  -  Terrain  mapping  product  questionnaire 

Attachment  6  -  Autonomous  Vehicle  questionnaire 
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Operational  Mission  Profile 


Surface 

RMS  Range  (in) 

% 

WNS 

IRI 

Other.. 

Primary  Roads 

0.1  to  0.3 

10% 

Secondary  Roads 

0.3  to  1.0 

20% 

Trials 

1.0  to  3.4 

30% 

Cross-Country 

1.5  to  4.8 

40% 

Duty  Profile/Mission  Cycle 

The  following  definition  describes  the  notional  MPCTD  duty  profile/mission  cycle.  Unless 
otherwise  specified,  performance  shall  be  demonstrated  on  surfaces  such  that  10%  is  completed 
on  Primary  Roads,  20%  on  Secondary,  30%  on  Trails,  and  40%  Cross-Country.  The  DoD  has 
defined  mission  profile  duty  cycle  percentages  and  RMS  values  for  surface  roughness.  The 
wave  number  spectrum  (WNS)  formulas  are  based  on  the  following  example. 

WNS  Formula: 

Gxx(n)  =  1.4  X  lO-V)'^  *^ 

Where: 

Gxx(n)  =  spectral  of  the  road  elevation  in  ft2/cycle/ft 
n  =  wave  number  in  cycle/ft 

1.4  X  10'^  =  roughness  coefficient  (amplitude  of  spectrum  at  1  cycle/ft) 

■2  9  =  slope  of  the  wave  number  spectrum. 

Note:  The  random  roughnesses  expressed  through  the  straight-line  wave  number  spectrum 
relationships  are  average  values  and  actual  road  roughness  will  naturally  contain  variability.  The 
upper  and  lower  limits  for  the  random  portion  of  the  road  roughness  have  a  +!-  3  dB  envelope. 


•  Primary  Roads 

There  are  four  types  of  primary  roads:  high  quality  paved,  secondary  pavement,  rough 
pavement,  and  highly  degraded  pavement.  All  may  consist  of  two  or  more  lanes,  all  weather, 
maintained,  hard  surface  (paved)  roads  with  good  driving  visibility  used  for  heavy  and  high 
density  traffic.  These  roads  generally  have  lanes  with  a  minimum  width  of  108  inches,  road 
crown  to  two  (2)  degrees  and  the  legal  maximum  GVW/GCW  for  the  county  and  state  is  assured 
for  all  bridges,  (a)  High  quality  paved  roads  are  typified  by  rural  US  interstates,  (b)  Secondary 
pavement  can  include  degraded  concrete,  macadam  concrete  or  asphalt  pavements  (small 
potholes,  alligator  cracking,  freeze/thaw  breakup),  (c)  Rough  pavement  consists  of  two  lane 
roads  with  degraded  shoulders,  and  marginal  subgrades  which  produce  long  wavelength  swells 
and  additional  degradation  of  the  surface,  (d)  Highly  degraded  pavement  consists  of  large 
potholes  in  various  states/quality  of  repair,  significant  surface  degradation,  and  marginal  to  poor 
subgrades. 
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Attachment  1 
Concept  Mission  Profile 


Surface 

Wave  Number  Spectrum 

RMS 

Roughness 

(inches) 

Average 

Speed 

%  Total 
Miles 

High  Quality  Paved  Road 

Gxxlnl^LdxlO-'ln)^** 

0.1 

65-75 

3% 

Secondary  Pavement  (Two  Lane  Paved 
Road) 

Gxx(n)=L9xlO-^(n)-2  5 

0.2 

55-65 

3% 

Rough  Pavement  (Degraded  Paved  Road) 

Gxx(n)=8.0x  10*"(n)-2  5 

0.3-0.5 

45-55 

3% 

Highly  Degraded  Pavement 

Gxx(n)=2.3xl0-5(n)-2  4 

0.5-0.7 

35-45 

1% 

•  Secondary  Roads 

There  are  three  types  of  secondary  roads:  loose  surface,  loose  surface  with  washboard  and 
potholes,  and  Belgian  Block.  These  roads  are  one  or  more  lanes,  all  weather,  occasionally 
maintained,  varying  surface  (e.g.,  large  rock,  crushed  rock  and  gravel)  intended  for  medium- 
weight,  low-density  traffic.  These  roads  have  no  guarantee  that  the  legal  maximum  GVW/GCW 
for  the  county  and  state  is  assured  for  all  bridges. 


Surface 

Wave  Number  Spectrum 

RMS 

Roughness 

(inches) 

Average 

Speed 

%  Total 
Miles 

Loose  Surface 

Gxx(n)=3.0x  10-5(n)-‘« 

0.3-0.6 

30 

8% 

Loose  Surface  with  Washboard  & 

Potholes*'^ 

Gxx(n)=4.0x  10-5(n)-2  4 

0.4-L2 

30 

10% 

Belgian  Block*^^ 

Gxx(n)=5.5x  10-5(n)-2  2 

0.3-L2 

20 

2% 

(1)  Loose  surface  with  washboard  roads  have  a  peak  amplitude  of  5.0x10'^  fd/cycle/ft  at  0.3  to  0.5  cycle/ft 
(2  to  3-foot  wavelengths).  Loose  surface  roads  with  a  high  density  of  potholes  have  a  peak  amplitude  of 
9.0x10’^  fd/cycle/ft  at  0.1  to  0.2  cycle/ft  (5  to  10  foot  wavelengths).  Generally,  washboard  occurs  in 
operational  areas  that  are  dry,  whereas  pothole  gravel  roads  occur  in  wet  operational  areas. 

(2)  Belgian  Block  secondary  roads  have  a  peak  amplitude  of  8.0  x  10-2  ftVcycle/ft  at  0.083  cycle/ft  (12  foot 
wavelengths)  and  these  wavelengths  are  180°  out-of-phase  left  to  right  which  produces  a  racking  input  to  the 
vehicle.  The  cobblestone  blocks  dominate  the  amplitude  of  the  wavelengths  at  1  cycle/ft. 

•  Trails 

One  lane,  unimproved,  seldom  maintained,  loose  surface  roads,  intended  for  low  density  traffic. 
Trails  have  no  defined  road  width  and  can  include  large  obstacles  (boulder,  logs,  and  stumps) 
and  no  bridging. 


Surface 

Wave  Number  Spectrum 

RMS 

Roughness 

(inches) 

Average 

Speed 

%  Total 
Miles 

Trails  (A) 

Gxx(n)=2.6x  10  5(n)-2® 

LO-3.4 

10-20 

30% 

Trails  (B) 

Gxx(n)=4.6x  10-5(n)-2  2 
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•  Cross-Country  Terrain 

Vehicle  operations  over  terrain  not  subject  to  repeated  traffic.  No  roads,  routes,  well-worn  trails, 
or  man-made  improvements  exist.  (This  definition  does  not  apply  to  vehicle  test  courses  that  are 
made  to  simulate  cross-country  terrain.)  In  addition,  cross-country  terrain  can  consist  of  tank 
trails  with  crushed  rock  or  having  large  exposed  obstacles  (rocks,  boulders,  etc.). 


Surface 

Wave  Number  Spectrum 

RMS 

Roughness 

(inches) 

Average 

Speed 

%  Total 
Miles 

Cross-Country^^* 

Gxx(n)=9.2x  10'‘(n)-2‘ 

1.5-4.8 

10-20 

40% 

(1)  Road  Left  and  Right  Track  Correlation.  Fixed  frequency,  RMS,  and  half-round  obstacles  shall  include 
roughness  or  events  where  the  left  and  right  wheel  paths  are  shifted  longitudinally  up  to  +/-  45  degrees 
(approximately  6  1/2  ft  (2m)). 


Definitions; 


Road  Roughness 

Spectral  characteristics  of  road  surface  measured  and  analyzed  in  terms  of  wave-number  spectra, 
rms,  IRI,  or  other  suitable  metric. 

Root  Mean  Squared  (RMS) 

A  measurement  used  to  describe  the  roughness  of  a  terrain. 

Washboard  Effect 

A  periodic  component  in  space  that  appears  in  the  wave  number  spectrum  as  a  sharp  peak  at  a 
wave  number  corresponding  to  the  reciprocal  of  the  “washboard”  wavelength.  Generally, 
washboard  roads  occur  in  operational  areas  that  are  dry. 

Wave  Number  Spectrum 

Wave  number  spectrum  epresents  road  roughness  data  as  a  straight-line  relationship  on  a  log-log 
plot  with  ft2/cycle/ft  on  the  y-axis  (wavelength  in  feet  or  spatial  frequency  of  the  distance 
between  the  bumps).  It  is  a  technique  for  measuring  and  monitoring  long  sections  of  various 
terrain  types,  including  paved  roads  and  off-highway  durability  test  courses,  that  can  be  used  to 
describe  all  potential  deployment  areas  of  a  vehicle.  Wave  number  spectrum  provides  a  vehicle 
and  speed  independent  measure  of  the  roughness  of  a  road. 

Typical  Soil  Parameters  for  Consideration 

•  Kc,  kf  and  n  (Bekker) 

•  C,  Phi  and  K  (modulus  of  deformation  for  shear) 

•  %  Compaction 

•  Density 

•  Moisture 

•  Depth  and  layering 

•  Surface  coefficient 
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Attachment  1 
Concept  Mission  Profile 


•  Soil  Impedance 

•  Bulk  Density 

•  Bearing  Capacity 

•  Proctor 

•  Cone  Index 

•  Soil  constitutive  model  parameters 

•  Others 


When  the  tire,  track,  vehicle,  and  terrain  data  are  combined  within  a  physics  based  model  then  the 
following  simulation  and  validation  approach  is  anticipated. 

3D  Terrain  contact  model 

o  Historically  used  for  off  Road  Courses  and  Bumps 
o  Road  Modeled  With  Triangular  or  other  Elements  (Like  FEA  model) 
o  Tire  Deflection  Calculated  As  “Weighted  Average”  Based  on  Volume  of 
Penetration  Into  Each  Element 
o  Includes  Tire  Carcass  Shape  Effect 
o  Fidelity  over  obstacles  with  enveloping, 
o  Frequency  of  road  input 


Such  a  simulation  environment  would  provide  high  fidelity  contact  force  generation  on  any  type 
of  3D  terrain  profile  and  it  would  be  possible  to  input  tread  pattern  and  develop  detailed  contact 
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force  distribution  on  the  terrain  surface. 


The  tire  and  track  models  could  be  validated  both  based  on  laboratory  measurements  and  full  vehicle 
sytem  measurements.  This  requires  the  ability  to  interact  with  rigorous  models  (FEA)  which  may  be 
developed  during  tire  and  band  track  design 


Physical  Test  Rig 


Virtual  Test  Rig  (Model) 


The  tire  and  track  soil  interface  simulations  have  been  developed  with  varying  levels  of  fidelity  and 
success.  Tire  tread  and  rubber  compound  can  be  dominant  parameters  when  predicting  tractive  effort  on 
slippery  surfaces  including  ice  and  snow.  Correlation  between  the  terrain  element  and  individual  tire  or 
track  element  is  often  critical  to  successful  simulation  as  indicated  below 
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Attachment  1 
Concept  Mission  Profile 


Viewport:  1  ODB:/h«mefti4ra6^k/1lretetTalrv6xa1t2D1.odb 
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Tracked  Terrain  Simulation  Environment 

•  Approach 

o  Detailed  track  model 

■  All  track  elements  included 

■  Bushings 

■  Grousers 

■  Single  Pin/Double  Pin/Rubber  track 

■  Suspension 

■  Track  tensioner 

■  Driven  by  contact  force  with  sprocket 
o  Terrain  Material  Model 


Model  Setup 


Actual 
hook 
location 


The  pintle 
hook  location 
in  the  model 


Gradually 
increasing 
drawbar  pull 
force 


Simulate  at  different  levels  of  fidelity 

o  Detailed  for  tractive  effort  and  soil  interaction  -  includes  soft  soil  model 
o  Simplified  “string  track”  model  for  low  freq  events 
Validate 

o  Tractive  effort 
o  Ride  quality 


#■ 

■ 
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Attachment  1 
Concept  Mission  Profile 

o  Discrete  bump  events 
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The  following  are  considered  the  minimum  subsystem  representations  (based  on  the  existing  NRMM) 
within  the  simulation  to  provide  results  which  can  successfully  trend  or  compare  performance  between 
vehicle  configurations.  Of  interest  is  how  your  simulation  environment  can  accommodate  these 
parameters  and  how  these  parameters  may  be  enhanced  or  integrated  into  a  more  accurate  simulation 
environment. 

Powertrain  Information 

Tractive  effort  vs.  speed  curve 

Engine  characteristics  (type,  displacement,  number  of  cylinders,  max  torque) 

Engine  speed  versus  engine  torque 

Total  net  engine  power,  each  engine 

Engine  to  torque  converter  gear  ratio  and  efficiency 

Aerodynamic  Characteristics 
Drag  coefficient 

Erontal  area 

Hydrodynamic  drag  coefficient 

Maximum  Braking  Coefficient 
Swim 

Combination  vehicle  draft 

-  Combination  fording  depth 

Vehicle  swamp  angle  during  egress 
Vehicle  swamp  angle  during  ingress 
Maximum  fording  speed 

Maximum  swim  speed  with  auxiliary  propulsion 

Suspension  Characteristics 

Spring  force/deflection  curve(s) 

Damper  force/velocity  curve(s) 

Jounce  and  Rebound  stop  location  and  rates 

Suspension  geometry  including  gross  motion,  travel  etc. 

-  Track  system  spring  rates 

Suspension  Design 

Tracks  vs.  Wheels 

Bogie/walking  beam/independent/hard  mounted 
Driver’s  seat  location/suspension  (spring  and  damper) 

Driver’s  mass 


Chassis 

Maximum  pushbar  force  vehicle  can  withstand  overriding  vegetation  stems 
Steering 

Vehicle  minimum  turning  radius  Left  and  Right,  Case 
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Attachment  2 

NRMM  Data  Input  Requirements 

General  Vehicle  Characteristics 

-  GVW 

Pitch  mass  moment  of  inertia 

CG  measurement 

All  vehicle  system  dimensions 

Settled  body  angle  relative  to  ground 

Wheel  (or  roadwheel)  and  Chassis  Characteristics 
Wheel  (or  roadwheel)  diameter  and  mass 

Wheel  (or  roadwheel)  longitudinal  position 

Wheel  (or  roadwheel)  force/deflection/damping 

Wheel  (or  roadwheel)  weight  (vehicle  weight  hy  wheel  position) 

Is  wheel  driven  /  hraked  force  distribution 

Contact  path  dimensions 

Tire  deflection  at  relevant  central  tire  pressure  settings 
Maximum  tire  speed  limit  for  each  deflection  scenario 
Tire  stiffness  at  each  pressure;  Tire  /  Track  revolutions  per  mile 

Unique  info  for  Tracked  Vehicles 

Drive  sprocket/idler  characteristics 

Information  for  track  model  (uniform  tension/local  tension/interconnecting  spring  models) 
Track  width 

Length  of  track  on  ground  (in) 

Grouser  height 
Maximum  allowable  sinkage 
Track  tension  (Ibf) 

Track  tensioner  spring  /  damping  rate 
Track  shoe  contact  areas 

Damping  coefficient  for  each  Sprocket  or  idler  assembly 

-  Track  grouser  height  for  each  assembly 

Geometry 

Belly  Geometry 

Horizontal  distance  from  CG  to  rear  axle  of  prime  mover 
Minimum  ground  clearance 
Driver’s  eye  height  above  ground 
Vehicle  projected  frontal  area 
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Vehicle  maximum  height  including  all  external  fixtures 

Vehicle  minimum  height  (excluding  vertical  perfusions,  fixtures,  etc.)  minimum  overhead 
clearance  requirement 

Length  of  each  vehicle  unit  (from  connection  point  to  connection  point) 

Pitch  mass  moment  of  inertia  about  the  CG  of  sprung  mass  (ih-sec^-in) 

Mobility  Performance!  pass  vehicle  cone  index  for  fie  grained  soils  for  each  assembly. 
Vehicle  lateral  stability 

Vehicle  absorbed  ride  quality  at  various  locations 
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Attachment  3 

General  Physics  Based  Model  Data  Input  Requirements 

The  following  represents  subsystem  data  which  is  anticipated  as  required  to  support  a  high  fidelity  high 
granularity  simulation  environment.  It  is  assumed  that  the  lower  fidelity  simulation  environment  would  be 
a  subset  of  the  high  fidelity  simulation  environment. 

Generalized  Data  Input  for  Powertrain  Model  Template 

Engine 

Mass  and  inertia  properties 
CG  location 
Location  of  all  mounts 

Stiffness  (force  versus  displacement)  curves  for  mounts  in  all  directions 
Damping  (force  versus  velocity)  curves  for  mounts  in  all  directions 
Data  for  engine  torque  as  a  function  of  rpm  and  throttle  position 
Engine  braking  data  (if  desired) 

Data  for  accessory  loads  on  engine  (AC,  fan,  alternator,  etc.) 

Idle  and  maximum  rpm 

Torque  Converter 

Mass  properties 

Characteristics  curves  for  performance  (i.e.  torque  and  speed  ratio  curves) 

Transmission 

Mass  properties 

Location  of  all  mounts 

Stiffness  (force  versus  displacement)  curves  from  mounts  in  all  directions 

Damping  (force  versus  velocity)  curves  for  mounts  in  all  directions 

Number  of  gears  and  gear  ratios 

Shift  profiles  (up  and  down  shift) 

Efficiency  (or  loss  data) 

Transfer  Cases  and  Differentials 
Mass  properties 

Location  of  all  mounts 

Stiffness  (force  versus  displacement)  curves  for  mounts  in  all  directions 
Damping  (force  versus  velocity)  curves  for  mounts  in  all  directions 
Gear  ratio 

Eunctional  description  (i.e.,  open,  biased,  locking,  etc.) 

Eunctional  data  (depending  on  above  description) 

Drive  Shafts  and  Elalf  Shafts 
Mass  properties 
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Hubs 


Mass  properties 

Gear  Ratio  (if  geared  hub  is  used) 

Generalized  Data  Input  for  Suspension  Model  Template 

Actual  data  required  depends  on  suspension  type.  The  example  below  is  for  SLA  independent  using 
conventional  spring  and  damper. 

Hard  Points 

Upper  control  arm  (front,  rear  and  outer) 

Lower  control  arm  (front,  rear  and  outer) 

Bumpstop  (upper  and  lower) 

Rebound  stop  (upper  and  lower) 

Spring  mount  (upper  and  lower) 

Shock  mount  (upper  and  lower) 

Tie  rod  (inner  and  outer) 

Wheel  center 

Drive  shaft  (inner  and  outer) 

Subframe  (front  and  rear) 

Anti-roll  bar 

Mass  properties  for  all  components  (weight,  CG,  mass  moments  of  inertia) 

Control  arms 

Spindles 

Half  shafts 

Springs 

Shocks 

Subframe 

Tie  rod 

Anti-roll  bar 

Bushings 

Bushings 

Define  bushing  orientation  and  preload 
Translational  stiffness  curve 
Rotational  stiffness  curve 
Translational  damping  curve 
Rotational  damping  curve 
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Attachment  3 

General  Physics  Based  Model  Data  Input  Requirements 

Dampers 

Force  versus  velocity  curves 

Springs 

Define  installed  length  or  preload 
Force  versus  displacement  curves 

Generalized  Data  Input  for  Tire  Model  Template 

Actual  data  required  depends  on  the  specifics  of  the  tire  model  employed 
Geometric  Properties 

Tire  section  width 

Tire  aspect  ratio 

Rolling  radius 

Contact  area  (footprint)  as  a  function  of  inflation  pressure  and  load/deflection 

Rim  width 

Rim  diameter 

Tread  depth 

Other 

Mass  and  Stiffness  Properties 

Wheel  end  assembly  weight 

Center  of  Gravity 

Mass  moment  of  inertia 

Load  deflection  curve 

Vertical  stiffness 

Lateral  stiffness 

Longitudinal  stiffness 

Cornering  stiffness 

Slip  characteristics 

Other 

Generalized  Data  Input  for  Track  Model  Template 

Geometric  Properties 
Track  width 

Track  contact  length 

Track  design  (i.e.,  single  pin,  double  pin,  rubber) 

Sprocket  radius 
Grouser  height 
Grouser  pitch 
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Area  of  the  shoe 
Roadwheel  radius 
Radius  of  the  idler 
Roadwheel  spacing 
Other 

Mass  and  Stiffness  Properties 
Roadwheel  height 

Mass  moment  of  inertia 

Initial  track  tension 

Suspension  design  (arms,  springs,  dampers,  etc.) 
Bushings 

Simplified  track  model 
o  String  track 
o  Track  superelement 
o  Other 

Soil  Model 

Data  input  depends  on  soil  constitutive  model 
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Attachment  4 

Vehicle  Mobility  Model  Product  Questionnaire 


1 .  Does  the  solver  support  parallel  processing  and/or  other  High  Performance  Computing  environment? 
If  so,  how  well  does  the  solution  time  scale  when  going  from  2  to  1000  cores?  Does  the  software  run 
on  both  Windows  and  Linux? 

2.  Is  the  modeling  environment  compatible  with  the  legacy  empirically  based  NATO  Reference 
Mobility  Model? 

3.  Does  the  interface  provide  a  simplified  “non-expert”  user  interface?  if  so,  describe  the  functionality. 
As  associated  to  non-expert  versus  expert  usage,  does  the  environment  allow  for  a  reduced  fidelity 
approach  which  substantially  reduced  run  time?  Does  the  non-expert  interface  verify  that  the  user 
enters  valid  data? 

4.  Can  three  dimensional  terrain  (i.e.  rough,  slopes,  sideslopes)  surfaces  be  simulated?  How  are  they 
defined?  Can  CIS  data  be  utilized?  If  so  what  format  is  required? 

5.  Is  an  off-road  tire  model  available?  If  so,  what  frequency  range  is  it  valid  for?  Describe  the  tire 
model,  including  ability  to  discern  contact  patch  size  and  pressure.  Can  a  custom  tire  model  be 
implemented?  If  yes,  how 

6.  Can  tracked  vehicles  be  modeled?  Describe  capabilities  for  building  the  tracks,  suspension  elements, 
track  tensioning,  etc.  Is  there  an  option  for  both  detailed  track  models  and  fast  running  track  models 
such  as  a  string  track  or  track  super-element?  Can  the  model  differentiate  between  single  pin,  double 
pin,  and  rubber  tracks? 

7.  Does  the  model  support  a  template  based  approach?  If  so,  describe  how  this  is  implemented.  What  is 
included  in  a  template?  How  are  the  templates  created  and  modified? 

8.  Does  the  model  support  deformable  bodies?  If  so,  does  it  support  ANCF  (absolute  nodal  coordinate 
formulation).  Does  it  provide  a  modal  approach  for  complex  flexible  bodies?  Is  there  an  internal 
finite  element  solver?  Is  there  an  ability  to  include  material  and  geometric  non-linearities  either 
through  an  internal  non-linear  finite  element  solver  or  via  co-simulation  with  external  non-linear 
finite  element  solvers? 

9.  Can  advanced  control  systems,  including  digital  discrete  multi-rate  controllers,  be  included  in  the 
simulation?  If  so,  describe  the  approach. 

10.  Does  the  modeling  approach  allow  for  contact  between  the  vehicle  and  the  terrain  other  than  the  tires 
or  tracks?  If  this  is  possible,  how  is  the  contact  modeled?  How  is  the  terrain  and  hull  geometry  for 
contact  modeled?  Describe  the  approach  and  capabilities. 

1 1 .  Describe  the  level  of  detail  included  in  the  power  train  and  driveline  model. 

a.  Are  the  engine  dynamics  modeled?  Describe  the  approach  taken.  How  are  engine  losses  and 
accessory  loads  accounted  for?  How  is  the  engine  integrated  with  transmission  designs?  Can 
Transmissions  ranging  from  manual  to  automatic  to  continuously  variable  to  infinitely 
variable  be  considered? 

b.  Is  there  an  ability  to  model  hybrid-electric  drives?  What  is  the  modeling  approach? 

c.  Is  the  torque  converter  explicitly  included?  How  is  it  modeled  and  what  data  is  required? 

d.  Are  the  differentials  and  transfer  cases  explicitly  modeled?  Can  features  such  as  differential 
locking,  clutches,  and  torque  biasing  be  included? 

e.  Can  the  driveline  be  configured  to  support  all-wheel  drive  on  multi-axle  vehicles? 

12.  Is  a  simulated  driver  included?  Does  the  driver  control  throttle,  brake,  clutch,  steering,  and  shifting? 
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13.  Is  the  driver  open  loop  or  closed  loop?  If  it  is  closed  loop,  describe  the  control  approach.  Can  it 
perform  realistic  human  driver  inputs,  for  example  to  determine  end  limits  on  a  double  lane  change 
maneuver? 

14.  Describe  how  a  “unique”  suspension  design  would  be  modeled.  Can  it  be  modeled  by  a  user,  or  does 
it  require  custom  code  development? 

15.  Can  deformable  terrain  be  included  in  the  model?  If  so,  describe  the  modeling  approach  and  data 
input  requirements,  and  how  the  model  is  applicable  for  tractive  effort  evaluation  and  soft  soil  grade 
climb  simulations.  Can  the  model  discern  between  soil  types,  such  as  coarse  grain  dry  sand  (S  per 
uses  classification)  and  fine  grain  (CH/CL  per  USGS  classification),  peat,  layered  soil,  various 
snow  conditions,  etc. 

16.  Can  the  tire-terrain  or  track-terrain  contact  support  FEA/DEM  for  deformable  terrain  at  the  contact 
patch/nodes? 

17.  Can  the  model  include  hydrodynamic  forces  as  might  be  encountered  by  a  vehicle  in  a  fording  event? 
How  are  the  forces  computed?  Can  the  model  be  used  to  predict  the  ability  of  a  vehicle  to  transition 
from  water  to  a  bank  or  ramp? 

18.  Will  the  model  support  hardware  in  the  loop  simulations?  If  so,  describe  specific  hardware/software 
requirements. 

19.  Can  the  model  be  used  to  calculate  fuel  economy  over  a  desired  mission  profile,  which  may  include 
grades,  rough  terrain,  obstacles,  deformable  soil,  weather  scenarios,  and  variable  speeds?  If  so, 
describe  the  approach  and  data  requirements. 

20.  How  is  the  software  licensed?  If  multiple  software  modules  exist,  define  what  is  needed  to  perform 
vehicle  mobility  simulations  including  control  systems,  flexible  bodies,  tires,  driver,  and  deformable 
surfaces. 

21.  What  is  the  software  cost?  Is  it  available  for  both  purchase  and  lease?  Is  a  short  term  or  on -demand 
lease  available? 

22.  Is  there  an  existing  capability  for  worldwide  training  and  support?  If  so,  describe.  Where  is  the 
training  performed?  How  is  technical  support  provided? 

23.  Describe  the  post  processing  capabilities  for  creating  animations  and  plots,  and  for  performing  data 
analysis.  Can  animations  (movie  files)  be  created  and  exported?  Can  simulated  test  data  be  imported 
for  cross  plot  and  correlation?  Can  frequency  domain  calculations  (EET  and  PSD)  be  performed? 

24.  What  is  the  current  version  of  the  software,  and  when  was  it  released?  When  is  the  next  planned 
software  release?  Will  the  next  release  feature  new  capabilities  applicable  to  ground  vehicle  mobility 
simulation?  If  so,  please  describe. 

25.  In  user  support  provided  in  the  licensing?  Describe  the  extent  of  user  support  and  how  it  is  obtained. 

26.  How  does  your  software  support  evaluation  of  uncertainty  in  model  parameters?  Are  stochastic 
methodologies  built  in?  Are  design  of  experiment  (DOE)  capabilities  included?  Describe  the 
capabilities. 
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Attachment  5 

Terrain  Mapping  Product  Questionnaire 
Product  Questionnaire 

1 .  Identify  the  types  of  terrain  data  used  in  the  simulation,  and  the  areal  extent  to  he  provided  along 
with  its  precision  and  fidelity? 

2.  Has  a  prototype  process  of  similar  integration  between  the  vehicle  modeling  environment  and 
GIS  been  developed  and  tested? 

3.  Has  a  production  version  of  item  2  been  developed  and  tested? 

4.  Is  the  process/software  currently  in  production  in  any  application?  If  so,  in  what  industry? 

5.  Is  the  data  currently  applicable  to  or  compatible  with  NRMM? 

6.  Is  support  documentation  currently  available  for  the  process/software  (white  paper,  etc.)? 

7.  Is  the  data  migration  process  easily  adaptable  through  built-in  scripting  and  API? 

8.  Are  the  data  capable  of  supporting  wide  ranges  of  coordinate  systems  and  projections  for  on-the- 

fly  projection? 

9.  Are  the  data  supported  in  a  wide  range  of  database  engines,  i.e.,  Microsoft  SQL  Server,  Oracle, 
IBM  DB2,  IBM  Informix,  Interbase,  Firebird,  Sybase,  PostgreSQL,  SQLite,  MSJET,  etc.? 

10.  What  kind  of  training  would  be  required  for  users  of  the  data  and  is  it  readily  supported? 

1 1 .  Do  you  provide  data  or  does  it  come  from  a  third  party  vendor? 

12.  Is  there  an  existing  customer  base  for  this  product?  Describe. 

13.  Does  the  process  support  import/export  of  CAD  or  other  modeling  data? 

14.  Are  the  process/data  OGC  compliant? 

15.  Are  the  data  predominantly  raster  or  vector? 

16.  Is  there  a  report-generating  component  in  the  program? 

17.  Are  the  geospatial  data  easily  adaptable  for  editing  and  customization  among  different  data  types 
and  software  platforms? 

18.  Is  there  sufficient  metadata  and  internal  data  description  to  support  linking  to  complex  look  up 
tables? 

19.  Will  the  data/process  support  import/export  from/to  modeling  and  simulation  software  platforms? 
Describe. 
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Simulations  of  Autonomous  vehicle  systems  require  unique  tool  capabilities  in  addition  to  those  identified  in 
the  previous  attachments.  Further,  autonomous  vehicles  have  a  broad  range  of  configurations  from 
walking/legged  systems  to  ultra-light  systems  intended  for  operation  inside  buildings  to  20,000  pound 
transport  vehicle  systems.  However,  the  systems  rely  on  similar  sensor  types  to  insure  successful  operation. 
As  such  any  speciality  solutions  to  support  autonomous  operations  should  be  described.  In  addition  to 
traditional  vehicle  dynamics  the  following  are  considered  in  support  of  the  analysis  of  autonomous  vehicle 
systems 


o  Can  the  simulation  environment  present  scene -based  operations  which  include  the 
challenges  associated  with  lit  and  unlit  conditions?  Can  the  environment  in  the 
simulation  be  impacted  by  fog  or  dust  or  other  environmental  conditions  which  can 
impact  sensor  performance?  Be  able  to  control  lighting,  fog  (that  can  effect  sensing) 
o  Can  the  objects  be  presented  as  3D  objects  with  variable  surfaces  and  surface 
coefficients? 

o  How  are  the  obstacles  represented  and  how  do  the  obstacles  react  during  loading,  are 
deformable  surfaces  included? 

o  Available  support  for  user  to  edit/sculpt  existing  terrain  data  sets? 
o  Be  able  to  support  dynamic  scenes,  i.e.  where  items  (iconic  pedestrians,  other 
vehicles)  are  moving  in  the  scene.  Intelligent  vehicles  will  need  to  be  able  to  detect 
and  avoid  static  as  well  as  such  moving  entities, 
o  Be  able  to  specify  textures  in  addition  to  geometry  for  objects 
o  Be  able  to  specify  reflectance  properties  (eg.  BRDF)  for  objects  needed  by  sensor 
models 

o  How  are  vision-based  sensors  represented,  what  are  the  metrics  for  performance? 

GPU  acceleration?  Ray  tracing? 
o  Are  terrain  data  sets  geo-referenced? 

o  Can  terrain  models  include  multiple  layers  including  large  low-res  and  hi-res  insets 
needed  for  sensors  and  sensor  performance  validation? 
o  Is  ephemeris  support  available  for  sun  and  satellite  positioning  for  comm  modeling? 
o  Is  there  an  ability  to  specify  map  data  such  as  locations  of  stop  signs,  traffic  signals 
etc.  Intelligent  vehicles  may  be  expected  to  follow  traffic  rules, 
o  Support  for  modeling  interiors  of  buildings  for  indoor  mobility  evaluation? 
o  How  are  the  inputs  from  the  sensors  applied  to  the  vehicle  simulation  and  what  is  the 
representative  control  system  update  rate. 
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APPENDIX  F  -THEME  5  RECOMMENDATIONS  FOR  A  VALIDATION 

EFFORT 


To  rapidly  complete  this  validation  effort  it  is  necessary  to  have  measured  vehicle  and  associated  test  data  to 
compare  against  the  predictions.  By  way  of  example,  data  from  a  capable  10-wheel  drive,  all-wheel  steer 
technology  demonstrator  vehicle,  developed  hy  the  Office  of  Naval  Research  (ONR)  and  the  US  Marine 
Corps,  was  made  available.  In  this  particular  case,  operational  test  data  had  been  developed  over  mission 
profile  representative  events.  Full  vehicle  dynamics  simulations  which  included  powertrain,  suspension,  tire 
soil  interaction,  etc.  had  been  developed,  thereby  establishing  that  sufficient  information  was  available  so  that 
accurate  models  over  events  of  interest  could  be  constructed. 

A  representative  photograph  and  prior  simulation  activities  of  the  vehicle  are  shown  below. 
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Vehicle  component,  powertrain,  tire  soil  interface,  tractive  force  slip,  and  other  parametric  data  necessary  to 
support  the  anticipated  level  of  accuracy  had  already  been  developed  and  could  he  provided  in  the  following 
representative  formats  to  assist  in  more  rapid  evaluation  of  the  available  tools  from  the  various  organizations. 

The  following  general  vehicle,  system,  and  subsystem  data  is  required  to  create  a  detailed  physics -based 
model  of  a  given  vehicle.  The  vehicle  selected  to  model  should  contain  modem  suspension  technology, 
powertrain,  limited  slip  differentials,  ABS  brakes,  electronic  control  systems  (traction  control,  stability 
control,  etc.). 

•  Full  Vehicle: 

-  mass  at  current  payload 

-  center  of  gravity  of  truck 

-  center  of  gravity  of  payload 

-  wheel  base 

-  track  width 

-  number  of  axles 

-  number  of  driven  axles 

-  traction  control  system 

A  typical  list  of  required  vehicle  and  component  input  data  is  provided  below. 

•  Powertrain: 

-  Engine: 
o  mass 

o  mass  moment  of  inertia  about  X,  Y,  and  Z  axes 
o  center  of  gravity  location  (X,  Y,  Z) 
o  rotating  mass  (crankshaft)  inertia 
o  location  of  all  mounts 

o  stiffness  (force  versus  displacement)  curves  for  mounts  in  all  directions 
o  damping  (force  versus  velocity)  curves  for  mounts  in  all  directions 
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o  data  for  engine  torque  as  a  function  of  rpm  and  throttle  position 
o  engine  braking  data  (if  desired) 

o  data  for  accessory  loads  on  engine  (AC,  fan,  alternator,  etc.) 
o  idle  rpm 
o  max  rpm 

-  Torque  Converter: 
o  mass 

o  mass  moment  of  inertia  about  X,  Y,  and  Z  axes 
o  center  of  gravity  location  (X,  Y,  Z) 

o  characteristic  curves  for  performance  (e.g.,  torque  and  speed  ratio  curves) 

-  Transmission: 
o  mass 

o  mass  moment  of  inertia  about  X,  Y,  and  Z  axes 
o  center  of  gravity  location  (X,  Y,  Z) 
o  location  of  all  mounts 

o  stiffness  (force  versus  displacement)  curves  for  mounts  in  all  directions 
o  damping  (force  versus  velocity)  curves  for  mounts  in  all  directions 
o  number  of  gears  and  gear  ratios 
o  shift  profiles  (up  and  down  shift) 
o  efficiency  (or  loss  data) 

-  Transfer  cases  and  differentials 
o  mass 

o  mass  moment  of  inertia  about  X,  Y,  and  Z  axes 
o  center  of  gravity  location  (X,  Y,  Z) 
o  location  of  all  mounts 

o  stiffness  (force  versus  displacement)  curves  for  mounts  in  all  directions 
o  damping  (force  versus  velocity)  curves  for  mounts  in  all  directions 
o  gear  ratio 

o  functional  description  (e.g.,  open,  biased,  locking,  etc.) 
o  functional  data  (depending  on  above  description) 

-  Drive  shafts  and  halfshafts 
o  mass 

o  mass  moment  of  inertia  about  X,  Y,  and  Z  axes 
o  center  of  gravity  location  (X,  Y,  Z) 

-  Hubs 

o  mass 

o  mass  moment  of  inertia  about  X,  Y,  and  Z  axes 
o  center  of  gravity  location  (X,  Y,  Z) 
o  gear  ratio  (if  geared  hub  is  used) 
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Data  Input  for  Suspension 


•  Suspension 

-  Hard  points  (X,  Y,  and  Z): 

o  upper  control  arm  (front,  rear,  and  outer) 
o  lower  control  arm  (front,  rear,  and  outer) 
o  bumpstop  (upper  and  lower) 
o  rebound  stop  (upper  and  lower) 
o  spring  mount  (upper  and  lower) 
o  shock  mount  (upper  and  lower) 
o  tie  rod  (inner  and  outer) 
o  wheel  center 
o  drive  shaft  (inner  and  outer) 
o  subframe  front  and  rear 
o  anti-roll  bar 

-  Mass  properties  for  all  components  (weight,  CG,  mass  moments  of  inertia) 
o  control  arms 

o  spindles 
o  halfshafts 
o  springs 
o  shocks 
o  subframe 
o  tie  rod 
o  anti-  roll  bar 
o  bushings 

-  Bushings 

o  define  bushing  orientation  and  preload 
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o  linear  stiffness  curve 
o  rotational  stiffness  curve 
o  linear  damping  curve 
o  rotational  damping  curve 

-  Dampers 

o  force  versus  velocity  curves 

-  Springs 

o  define  installed  length  or  preload 
o  force  versus  displacement  curves 
•  Deformable  Tire 

-  Operating  Conditions 
o  Inflation  Pressure 
o  Tread  Depth 

o  Ambient  Temperature 

-  Basic  Data  And  Geometry 
o  Tire  Section  Width 

o  Tire  Aspect  Ratio 
o  Rim  Diameter 
o  Load  Index 
o  Speed  Symbol 
o  Rim  Width 
o  Rolling  Circumference 
o  Tire  Mass 
o  Belt  Width 
o  Tread  Width 
o  Interior  Volume 
o  Belt  Lat  Curvature  Radius 

-  Static  and  modal  data  for  each  inflation  pressure 
o  Tire  Long  Stiffness 

o  Tire  Lat  Stiffness 
o  Tire  Tors  Stiffness 
o  Tire  Long  Stiffness  Progr 
o  Tire  Lat  Stiffness  Progr 
o  Cornering  Stiffness 
o  Pneumatic  Trail 
o  Camber  Stiffness 
o  Belt  Lat  Bend  Stiffness 
o  Belt  Rad  Torsion  Stiffness 
o  Belt  Torsion  Stiffness 
o  Belt  Twist  Stiffness 
o  Belt  Torsion  Lat  Displ  Coupl 
o  Belt  Torsion  Twist  Damp 
o  Belt  Lat  Bend  Damp 
o  Rad  Dynamic  Stiffening 
o  Tang  Dynamic  Stiffening 
o  Time  Const  Dynamic  Stiffening 
o  Radial  Hysteretic  Stiffening 
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o  Radial  Hysteresis  Force 
o  Tang  Hysteretic  Stiffening 
o  Tang  Hysteresis  Force 
o  Belt  Extension  At  Vmax 
o  Rel  Long  Belt  Memb  Tension 
o  Rel  Long  Belt  Memb  Tension  Red 

-  Tread  Properties 
o  Tread  Depth 

o  Tread  Base  Height 
o  Rel  Min  Tread  Shoulder  Height 
o  Rel  Tread  Shoulder  Width 
o  Stiffness  Tread  Rubber 
o  Stiffness  Progr  Tread  Rubber 
o  Tread  Positive 

o  Tread  Pattern  Shape  Lactor  Tang 
o  Tread  Pattern  Shape  Lactor  Long 
o  Lat  To  Long  Tread  Stiffness  Ratio 
o  Sidewall  To  Tread  Stiffness  Ratio 
o  Damping  Tread  Rubber 
o  Max  Lriction  Velocity 
o  Sliding  Velocity 
o  Blocking  Velocity 
o  Low  Ground  Pressure 
o  Med  Ground  Pressure 
o  High  Ground  Pressure 
o  Mu  Adhesion  At  Low  P 
o  Mu  Max  At  Low  P 
o  Mu  Sliding  At  Low  P 
o  Mu  Blocking  At  Low  P 
o  Mu  Adhesion  At  Med  P 
o  Mu  Max  At  Med  P 
o  Mu  Sliding  At  Med  P 
o  Mu  Blocking  At  Med  P 
o  Mu  Adhesion  At  High  P 
o  Mu  Max  At  High  P 
o  Mu  Sliding  At  High  P 
o  Mu  Blocking  At  High  P 
o  Time  Const  Tire  Heating 
o  Time  Const  Tread  Heating 
o  Tire  Temp  At  Ref  Slip  Low  V 
o  Tread  Temp  At  Ref  Slip  Low  V 
o  Tread  Temp  At  Ref  Slip  Med  V 
o  Tread  Temp  At  Ref  Slip  Vmax 
o  Temp  Ref  Slip 
o  Perc  Lrict  Power  Heating  Tread 
o  Wear  Rate  Coefficient 
o  Wear  Rate  Exponent 

-  Tire  Imperfections 
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o  Static  Balance  Weight 
o  Static  Balance  Ang  Position 
o  Dynamic  Balance  Weight 
o  Dynamic  Balance  Ang  Position 
o  Radial  Non  Uniformity 
o  Radial  Non  Unif  Ang  Position 
o  Tang  Non  Uniformity 
o  Tang  Non  Unif  Ang  Position 
o  Ply  Steer  Percentage 
o  Conicity 
o  Run  Out 

o  Run  Out  Ang  Position 
-  Control  Tire  Inflation  System 
o  Inflation  Pressure 
o  Inflation  Pressure  2 
o  Cleat  Width 
o  Rim  Inertia 


Typical  characteristics  required  for  soil  properties  simulations  include: 

•  Liquid  limit 

•  Plastic  limit 

•  Moisture  content 

•  Density 

•  Particle  size  distribution 

•  Soil  shear  properties 

In  addition,  detailed  terrain  data  and  the  measured  vehicle  response  in  terms  of  traction,  acceleration,  ride 
quality,  stopping  distance,  stability,  etc.,  had  been  quantified  over  conditions  similar  to  those  indicated  below. 
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TRAILS 


CROSS-COUNTRY 


SAND 


EMBEDDED  ROCK 


CLAY 


LOAM/SILT 


Of  particular  interest  is  the  ability  of  the  potential  vehicle  dynamics  tools  to  accurately  predict  speed  and  ride 
quality  and  damaging  energy  to  the  vehicle.  Historically  NRMM  only  considered  “half’  the  vehicle  and 
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therefore  all  of  the  ride  quality  test  conditions  required  that  the  humps  he  identical  under  both  left  and  right 
wheel  path.  Current  vehicle  and  analysis  technology  provides  for  substantially  improved  ride  quality  over 
complex  terrain  and,  therefore,  representative  terrain  roughness  moving  away  from  the  traditional  RMS  and 
toward  WNS  conditions  would  be  used  for  the  evaluation  of  the  various  solutions. 


TRADITIONAL 


TRADITIONAL 


NATURAL 


Available  Test  and  Simulation  Events 

The  committee  was  briefed  that  data  for  the  following  events  was  available.  During  that  discussion,  it  was 
recognized  that  requiring  too  many  events  during  this  basic  evaluation  stage  could  require  too  much  time  and 
cost  to  accomplish  the  evaluation.  The  discussion  identified  that  results  were  desired  in  approximately  6 
months. 
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Constant  Radius 

Speed  on  5%  Grade 

Double  Lane  Change  -  Paved  &  Gravel  Surface 

Straight  Line  Acceleration 

Road  Departure  Recovery 

Straight  Line  Braking 

30%  -  40%  Side  Slope  Slalom 

Washboard  Event 

Mission  Profile  Trails  &  Cross  Country 

Traditional  RMS  Course 

30%  Dry  Sand  Grade 

Traditional  WNS  Course 

40%  Dry  Sand  Grade 

Tractive  Effort 

24-36-inch  Vertical  Step 

Vehicle  Cone  Index  (VCI) 

Discrete  Events  (Potholes,  Speed  Bumps) 

MOUT  Rubble  Pile 

6-inch,  8-inch,  10-inch  Half-Rounds 

MOUT  Crater 

V-Ditch  Obstacle 

Based  on  the  discussions,  the  following  10  events  were  identified  as  appropriate  for  evaluation  of  the  potential 
solutions. 

•  Fundamental  Handling: 

-  Straight  Line  Acceleration 

-  Straight  Line  Braking 

-  Constant  Radius 

-  Double  Lane  Change  -  Paved  &  Gravel  Surface 

•  Deformable  Surfaces: 

-  30%  Side  Slope  Slalom 

-  30%  Dry  Sand  Grade 

-  Tractive  Effort 

-  Vehicle  Cone  Index  (VCI) 

•  2-D  vs  3-D  Path  Track: 

-  Traditional  RMS  Course 

-  Traditional  WNS  Course 

The  top  three  scored  solutions  were  then  approached.  Two  of  the  top  three  indicated  that  results  could  be 
provided  within  the  6-month  time  frame  and  the  third  indicated  that  solutions  were  possible  within 
approximately  9  months.  However  it  was  identified  that  funding  would  be  required  to  all  of  the  potential 
providers  to  support  their  efforts.  The  funding  requirements  ranged  from  $200,000  to  $400,000  depending 
upon  the  number  of  organizations  chosen. 

With  this  baseline  established,  it  was  apparent  that  a  variety  of  solutions  are  available  from  commercial  and 
university  based  efforts.  Further,  it  was  apparent  that  if  the  necessary  vehicle  component  and  system  test  data 
are  available  it  is  possible  to  rapidly  and  cost  effectively  identify  capable  next  step  solutions.  Based  on 
subsequent  meetings  and  guidance  from  the  head  of  the  committee,  the  decision  was  made  to  forego  the 
interim  next  step  and  move  forward  to  the  more  formal  and  lengthy  Validation  and  Verification  process.  This 
activity  will  be  led  by  Theme  7. 
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